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Abstract. The authors show that a wide class of Fredholm determinants arising 
in the representation theory of "big" groups such as the infinite-dimensional unitary 
group, solve Painleve equations. Their methods are based on the theory of integrable 
operators and the theory of Riemann-Hilbert problems. 
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stands for the Gauss hypergeometric function, and z,z',w,w' 



are some complex numbers. We call K{x, y) the continuous 2-P1 kernel or simply 
the 2 -Pi kernel. 

The basic problem considered in this paper is the derivation of an ordinary 
differential equation for the Fredholm determinant D{s) = det(l — K\(^g .^^)). 

This kernel originates in the representation theory of the infinite-dimensional 
unitary group U{oo). Briefly, decomposition of a certain natural representation 
of U{oo) into irreducibles is described by a probability measure on the infinite- 
dimensional space of all irreducible representations; a projection of this measure 
onto a 1-dimensional subspace has the distribution function equal to D{s) = det(l — 
-^|(s,+oo)); where K is as above. The study of this representation theoretic problem 
is the main subject of the two recent papers [012], [B05]. For a more detailed 
description of the problem and the results in these papers the reader is referred to 
§1 below. 

The problem of deriving differential equations for determinants of the form D{s) 
as above, has a long history. In their pioneering work [JMMS] in 1980, M. Jimbo, 
T. Miwa, Y. Mori, and M. Sato considered the so-called sine kernel which has the 
form (0.1) with ip{x) = I/tt, A{x) = sinx, B{x) = cos a;. They showed that the 
determinant of the identity operator minus this kernel restricted to an interval of 
varying length s can be expressed through a solution of the Painleve V equation. 
Their proof was based on the theory of isomonodromy deformations of linear sys- 
tems of differential equations with rational coefficients. This theory in turn goes 
back to the work of Riemann, Schlesinger, Fuchs, Garnier, and others. [JMMS] 
used the results of [JMU] and [JM], where the theory of isomonodromy deforma- 
tions was developed in a setting more general than in the classic papers mentioned 
above. Along with the one interval case, [JMMS] also considered the restriction of 
the sine kernel to a union of a finite number of intervals. They showed that the 
corresponding Fredholm determinant, as a function of the endpoints of the inter- 
vals, is a T -function (in the sense of [JMU]) of the corresponding isomonodromy 
problem. In other words, it can be expressed through a solution of a "completely 
integrable" system of partial differential equations called the Schlesinger equations. 

Kernels of the form (0.1) are of great interest in random matrix theory. Indeed, 
the Fredholm determinant related to the kernel (0.1) restricted to a domain J, 
with A and B being nth and (n — l)st orthogonal polynomials with the weight 
function ij}, measures the probability of having no particles in J for certain n- 
particle systems called orthogonal polynomial ensembles. Such systems describe 
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the spectra of random unitary and Hermitian matrices. We refer the reader to 
[Mel] for details. 

The results of [JMMS] attracted considerable attention in the random matrix 
community. In 1992 M. L. Mehta [Me2] rederived the Painleve V equation for the 
sine kernel. Approximately at the same time, C. Tracy and H. Widom [TWl] gave 
their own derivation of this result. Moreover, they produced a general algorithm 
(see [TW4]) to obtain a system of partial differential equations for a Fredholm 
determinant associated with a kernel of type (0.1) restricted to a union of intervals, 
in the case that the functions ijj, A, B satisfy a differential equation of the form 

d_ 

dx 

where R{x) is a traceless rational 2x2 matrix. Using their method, they derived 
different Painleve equations for a number of kernels relevant to random matrix 
theory [TW1]-[TW4]. 

Shortly after, J. Palmer [Pal] showed that the partial differential equations aris- 
ing in the Tracy- Widom method are precisely the Schlesinger equations for an 
associated isomonodromy problem. 

Among more recent papers, we mention (in no particular order) the works 
[AsvM] , [AvM] , where a different approach to the kernels arising from matrix models 
can be found, the paper [HS] , where the Painleve VI equation for the Jacobi kernel 
was derived, the paper [DIZ2], where the theory of Riemann-Hilbcrt problems was 
applied to derive the Schlesinger equations for certain kernels and to analyze the 
asymptotics of solutions, the paper [HI], where a multidimensional analog of the 
sine kernel was treated using the isomonodromy deformation method, and the pa- 
pers [FW] , [WF] , [WFC] , where, in particular, a two-interval situation was reduced 
to an ordinary differential equation in one variable. 

Returning to our specific 2-^1 kernel, we find that our functions ip, A, B satisfy 
an equation of the form (0.2) (see Remark 4.8 below). 

However, the method in [TW4] leads in our case to considerable algebraic com- 
plexity, and we have not been able to see our way through the calculation. A similar 
situation arose in the case of the (simpler) Jacobi kernel, for which the method in 
[TW4] leads to a third order differential equation. This equation was shown to 
be equivalent to the (second order) Painleve VI equation only in the later work of 
Haine and Semengue [HS] . In the face of these difficulties, we decided to look for a 
different approach. 

The representation theoretic origin of the 2F1 kernel suggests a new approach. 
It turns out that the construction of the kernel K, see §1, strongly indicates that 
K should have a "simple" resolvent kernel L — K{1 — K)~^. "Simple" in the sense 
that the formula for L(a;, y) should not involve any special functions! At the formal 
level "det(l — K) = (det(l + L))~^". However, we are interested in the restricted 
operator -?^|(s,+oo)5 ciiid it is not at all clear that the "simple" kernel L can be used 
in any way to compute D{s) = det(l — _|_oo)). It is the basic observation of this 
paper that the kernel L can indeed be used to compute D{s), and this leads, as we 
will see, to the desired differential equations. 

In the analysis that follows, a crucial fact is that both kernels K and L are 
integrable in the sense of [IIKS] . We refer the reader to the Appendix for the def- 
inition and basic properties of integrable operators and also for the definition of a 
Riemann-Hilbert Problem (RHP). Our method is as follows (see §5). 
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Step 1. The kernel K{x,y) is expressed through an exphcit solution of a RHP 
(]R, v) , where v comes from L and is "simple" . 

Step 2. D{s) = det(l — ^^1(5, +00)) is expressed through the solution of a 
normalized RHP ((s, +cxd), v), where v involves special functions as in (0.1). 

Step 3. The product mgrn satisfies the RHP (M \ (s, +00) = {—oo,s],v), where v 
is again the "simple" jump matrix occurring in Step 1. 

Step 3, which is the key fact, is a consequence of the theory of integrable operators 
and the following elementary observation: let S = Ei U E2 C C be a union of two 
contours. Let m, mi be solutions of the RHP's (E,f), respectively. Then 

777-2 = mim~^ solves the RHP (E2, V2 = m+v~^m^^ = m_v~^ml^). Conversely, if 
m, 7772 are solutions of the RHP's (S, v), (S2, V2), then mi = solves the RHP 

As we will see, if v is the jump matrix associated via the theory of integrable 
operators with the kernel L , then m+v~^m'^^ = m-V~^mZ^ is the jump matrix v 
associated with the kernel K. 

In the RH framework, differential equations arc deduced from the fact that the 
jump matrix for the problem at hand can be conjugated to a form which does 
not depend on the parameters relevant for the problem. A prototypical calcu- 
lation, which can be traced essentially to the beginning of the inverse scattering 
theory, is as follows (see, e.g., [DIZl] and references therein). The defocusing Non- 
linear Schroedinger (NLS) equation is associated with the RHP (S = M, fa;,t = 

gi^aa ^g-i^aa) ^^ere 
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for some reflection coefficient r. If tti is a solution of (M, Va;^t), then = me^^'^^ 
solves the RHP (M, v) which is independent of x and t. It follows that ^ and ^ 
solve the same RHP and hence ^ and ^ have no jump across S = R. 
A short calculation then leads to the Lax pair ^ = ^ = for some 

polynomial matrices P = P{C), L = L{(). Cross-differentiation -§1^ ~ 
then leads to the NLS equation. 

As we will see in §4, the jump matrix v in Steps 1 and 3 is easily conjugated to a 
jump matrix V which is piecewise constant. In the spirit of the above calculation for 
NLS, this means that a solution M of the RHP M_|_ = M^V can be differentiated 
with respect to the variable ( on the contour, and also with respect to s, leading 
as above to the relations of the form ^ = PM, ^ = LM where P = P(C) and 
L = L{Q are now rational. Cross-differentiation then leads to a set of differential 
relations. In order to extract specific equations, such as PVI for D{s) = det(l — 
K\{8,+oo))^ we recall the result in [Pal]. As V is piecewise constant, the above 
equations = PM, = LM describe an isomonodromy deformation, and 
hence one can construct an associated tau-function r = r(s) as in [JMU]. A separate 
calculation (§6) shows that in fact D[s) = t(s), and PVI follows using calculations 
similar to those as in [JM, Appendix C]. 

The above calculations generalize immediately to the case where the interval 
(s, +00) is replaced by a union of intervals J. 

The idea of reducing the Riemann-Hilbert problem for r?7s to a problem with a 
piecewise constant jump matrix has been used recently in [Pal] , [HI] , [DIZ2] , [KH] , 
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see also [Its]. However, the method outhned above of performing the reduction 
seems to be new. 

As noted above, the property of the kernel K which is important for us, is the 
existence of a simple resolvent kernel L — K{1 — K)~^. This property seems to 
be new and was first observed in the context of the representation theory of the 
infinite symmetric group S{oo) in [BOl]. In random matrix theory the operators K 
which arise are projection operators (of Christoffel-Darboux type), or their scaling 
limits. All these kernels have norm 1 and hence the operator L = K{1 — K)~'^ is 
not defined. However, our problem has a different origin which makes it possible 
not only to define L, but also to express it in an explicit way [BOl], [B05]. 

The method that we introduce can be used to recover the results in [TW4] 
for integrable operators with entries satisfying equations of type (0.2). We will 
illustrate the situation in the specific case of the Airy kernel in §9. 

In the remainder of the paper we consider a variety of kernels similar to (0.1). 

Firstly, we apply our methods to the Jacobi kernel and we prove that the de- 
terminant of the identity minus the Jacobi kernel restricted to a finite union of 
intervals is the r-function of the corresponding isomonodromy problem. For the 
one interval case we again get the Painleve VI equation, reproving the result of 
[HS]. 

Secondly, we apply our formalism to the so-called Whittaker kernel and its special 
case - the Laguerre kernel. The Whittaker kernel appeared in the works [P.I-P. V] , 
[BOl], [Borl] on the representation theory of the infinite symmetric group. The 
calculations for the kernel are applicable to (the simpler case of) the Whittaker 
kernel. We prove that the Fredholm determinant of the Whittaker kernel on a union 
of intervals is a r-function of an isomonodromy problem, and we derive Painleve V 
in the one interval case. This last result was proved in [Tr], and in [TW4] for the 
special case of the Laguerre kernel. 

Finally, we observe that the 2F1 kernel degenerates in a certain limit to a kernel 
which we call the confluent hyper geometric kernel. This kernel appears in a problem 
of decomposing a remarkable family of probability measures on the space of infinite 
Hermitian matrices on ergodic components, see [B04]. It can also be obtained 
as a scaling limit of Christoffel-Darboux kernels for the so-called pseudo-Jacobi 
orthogonal polynomials, see [WF], [B04]. We show that the Fredholm determinant 
in the one interval case for this kernel can be expressed in terms of a solution of the 
Painleve V equation. The confiuent hypergeometric kernel depends on 1 complex 
parameter r, and for real values of r the last result was proved in [WF]. For r = 
the kernel turns into the sine kernel, which recovers the original result of [JMMS]. 

The paper is organized as follows. In §1 we describe the representation theoretic 
origin of the problem. In §2 we introduce the 2F1 kernel and study its properties. In 
§3 the resolvent kernel L is defined, and the matrix m in Step 1 above is considered. 
In §4 we derive the Lax pair for M as above. In §5 we describe the general setting in 
which our method is applicable. The reader interested primarily in the derivation 
of the differential equations might want to start reading the paper with this section. 
In §6 we prove that the Fredholm determinants of kernels, satisfying the general 
conditions of §5, are r-functions of associated isomonodromy problems. In §7 we 
solve our initial problem: the Painleve VI equation for det(l — ^oq)) is derived. 
§8 deals with the applications of our method to the Jacobi, Whittaker, and confluent 
hypergeometric kernels. §9 presents a general approach to kernels of the form (0.1) 
subject to (0.2), worked out in the case of the Airy kernel. Finally, the Appendix 
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contains a brief description of the formalism of integrable operators and Riemann- 
Hilbcrt problems. 

A discrete version of many of the results in this paper is given in [Bor3]. 
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1. Harmonic analysis on the infinite-dimensional unitary group 

By a character of a (topological) group K (in the sense of von Neumann) we 
mean any central (continuous) positive definite function x on K normalized by the 
condition x(e) = 1. Recall that centrality means xidh) = xi^g) for any g,h & K, 
and positive definiteness means Tlii j ^i^j x{9i9j ) > for any Zi ^ Qi & K, 
i= 1, . . . , n. The characters form a convex set. The extreme points of this set are 
called indecomposable characters, and the other points are called decomposable 
characters. 

The characters of K give rise to representations in two ways. 

Through the Gelfand-Naimark-Segal construction each character x determines 
a unitary representation of K which will be denoted as n(x). When x is indecom- 
posable, n(x) is a factor representation of finite type in the sense of von Neumann, 
see [Th]. Recall that n(x) is a factor representation means that if 5' commutes with 
{n(x)((7), g G K} and S lies in the weak closure W of {n(x)((7), g G K}, then S is 
a multiple of the identity. Finite type means that W carries a finite trace function. 

Alternatively (see [Oil]), set G = K x K and let diagK denote the diagonal 
subgroup in G, which is isomorphic to K. We interpret % as a function on the first 
copy of K in G, and then extend it to the whole group G by the formula 

'^{91,92) = x{9i92^), {91,92) e K. 

Note that i/j is the only extension of x that is a diag K-biinvariant function on G. 
The function i/j is also positive definite, so the GNS construction assigns to it a 
unitary representation which we will denote by T{x)- By its very construction, it 
possesses a distinguished diag i^-invariant vector. 

If X is indecomposable then T{x) — T{x^'^^) is irreducible. The representations 
of the form T(x) with indecomposable x's are exactly the irreducible unitary rep- 
resentations of the group G possessing a K-invariant vector. See [Oil] for details. 

If K is a finite or compact group then the indecomposable characters of K are 
all of the form 

x-te) = (1.1) 

dimTT 

where tt is an irreducible (finite-dimensional) representation of K, and dimyr is its 
dimension. Moreover, any character can be written in a unique way as a convex 
linear combination of indecomposable ones: 

X{9)= E ^WX'^(^), P{^)>0, E ^(^) = 1- (1-2) 

7r€lrr(K) 7reIrr(K) 



If X = X'^ is of the form (1.1) with an irreducible tt then n(x) = tt, and T{x) = 
7r®7f, where W denotes the representation conjugate to tt. If tt acts in V then tt^W 
acts in V^V* ~ End(y), and Id G End(y) is the diag i^-invariant vector for T(x). 

In particular, if = U{N), the group oi N x N unitary matrices, then the 
irreducible representations of K are parametrized by the highest weights (see, e.g., 
[Zh]) 

A= (Ai > ••• > Ajv), KeZ, i = l,...,N, 
and every character can be written in the form (1.2) 

X= Pn{X)x\ (1-3) 

Ai>-->Aiv 

where x^ is the normalized (as in (1.1)) character of U{N) corresponding to A. 
Note that the coordinates of A may be negative. 

Now let K = U{oo) be the infinite-dimensional unitary group defined as the 
inductive limit of the finite-dimensional unitary groups U{N) with respect to the 
natural embeddings U{N) > U{N+1). Equivalently, U{oo) is the groiip of matrices 
U = [uij]'^j^i such that all but finitely many off-diagonal entries are zero, all but 
finitely many diagonal entries are equal to 1, and U* = U~^. 

A fundamental result of the representation theory of the group U{oo) is a com- 
plete description of indecomposable characters. They are naturally parameterized 
by the points 

a;=(a+,/3+,a-,/3-,7+,7-)eK'°°+' 



such that 



at >a+ >■■■>(), > /32+ > • • • > 0, 
ar > «2" > ■ ■ ■ > 0' /3r > > ■ • • > 0, 



7+ > 0, 7" > 0, (1.4) 

oo 

J](«+ + /3+ + ar+/3-)<oo, p+ + P^<l. 

The values of extreme characters are provided by Voiculescu's formulas [Vo]. This 
classification result can be established in two ways: by reduction to a deep theorem 
due to Edrei [Ed] about two-sided totally positive sequences, see [Boy] and [VK], 
and by applying Kerov-Vershik's asymptotic approach, see [VK] and [OkOl]. 

We denote the set of all points u satisfying (1.4) by Cl. The coordinates af, (3f , 
a~ , 7+, 7~ are called the Voiculescu parameters. 

Instead of giving a more detailed description of the indecomposable characters 
(which is rather simple and can be found in [Vo] ) , we will explain why such param- 
eterization is natural. It can be shown that every indecomposable character of 
U{oo) is a limit of indecomposable characters x'^*'^'' of growing finite-dimensional 
unitary groups U{N) as ^ oo. Here X{N) = Ai(A") > • • • > AAr(A) is a highest 
weight of U{N). The label G O of the character x'^ can be viewed as a limit of 
A(A)'s as A — > oo in the following way. 

We write the set of nonzero coordinates of A (A) as a union of two sequences of 
positive and negative coordinates: 

{A,(A)^0} = A+(A)U(-A-(A)), 
A+(A) = {A+(A) > ■ ■ ■ > A+(A)}, A-(A) = {Ar(A) > • • • > A," (A)}, 



where > 0, A~ > for all i, and k and I are the numbers of positive and negative 
coordinates in A(A^), respectively. Note that k + l < N. We now regard A"'"(A'^) and 
X~{N) as Young diagrams (of length k and /, respectively), and write them in the 
Frobenius notation (see [Mac, §1] for the definition): 

X+{N) = {p+{N) >p+{N)>...\ q+{N) > q+{N) >...), 
A-(iV) = (p-iN) >p^{N)>...\ q-iN) > q^iN) >...), 

Then, if is a limit of x^^"^-* as A'" ^ oo, we must have 



= ito Elm, = £tM, 



(1.5) 



for all i — 1,2,..., see [VK], [OkOl]. The parameters 7+, 7" can also be de- 
scribed in a similar manner. Since we will not be concerned with them, we refer an 
interested reader to [VK], [OkOl] for the asymptotic meaning of 7"*" and 7". 

Observe that the condition + < 1 in (1.4) is now easily explained — it 
follows from the relation qf + q^ = k + l — 2<N. 

The next question that we address is how the characters of U (00) decompose in 
terms of the indecomposable ones. 

Theorem 1.1 [012]. Let x be a character ofU{oo). Then there exists a unique 
probability measure P on Q such that 



X - / X^Pxid^), (1-6) 



where x'^ is the indecomposable character ofU{oo) corresponding to co 

The measure is called the spectral measure of the character x- The problem 
of finding the spectral measure for a given character x is referred to as the problem 
of harmonic analysis for x- 

The decomposition (1.6) is the infinite-dimensional analog of (1.3). 

Since the indecomposable characters x"^ are limits of the normalized characters 
X^(iV) of U{N), it is natural to expect that the measure from Theorem 1.1 can 
be approximated by discrete measures Pn from (1.3) as A?" — > 00. To formulate the 
exact result we need more notation. 

Define Q° as the set of points uj° = (a"*", a", f3~) G satisfying the con- 
ditions (1.4). There is a natural projection 0^0° which consists of omitting the 
2 gammas. Denote by P° the push-forward of the measure P under this projection. 
As we will only be concerned with statistical quantities depending on u>°, and not 
on 7"'',7~, it is enough to consider P° instead of P. 

For every A^ = 1,2,... define a map ijv which embeds the set of all highest 
weights A(iV) of U{N) into n° as follows. For A(iV) = (Ai(iV) > ••• > Aiv(A^)), 
using the above notation, we set 
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Theorem 1.2 [012]. Let x be a character ofU{oo), xn be its restriction to U{N), 
and 

X\u{N) = 

be the decomposition of xn on indecomposable characters. Then the projection P° 
of the spectral measure of x is the weak limit of push-forwards of the measures 
Pn under the embeddings iN ■ In other words, if F is a bounded continuous function 
on 0.° , then 

lim V F{iN{\))PN{\) = [ F{u)P°{dw). 



N 



Ai>->A 



Now, following [B05] , we apply the above general theory to a specific family of 
decomposable characters of U{oo) constructed in [012]. The group U{oo) does not 
carry Haar measure, and hence the naive definition of the regular representation 
fails. The representations in [012] should be viewed as analogs of the nonexist- 
ing regular representation of U{oo). A beautiful geometric construction of these 
representations can be also found in [012] . 

For every N = 1,2,... and a highest weight A = (Ai > • • • > Ajv) set 

N 



Piv(A) = CAT ■ DimliX) ■ H /(A, - i), 



i=l 

^^^^ ^ r{z - x)r{z' - x)r{w + n + i + x)r{w' + n + i + x)' ^^'^ 

^r{z + w + i)r{z + w' + i)r{z' + w + i)T{z' + w' + i)r(i) 

CN = [[ 



i=l 



T(z + z' + w + w' + i) 



where Dim7v(A) is the dimension of the irreducible representation of U{N) corre- 
sponding to A, 

1 — I 

i<i<j<N ■' 

see, e.g., [Zh]. Here ^, w, w' are complex parameters such that Pn{X) > for all 
N and A. This implies that 

(1) z' = z e C\Z OT k < z, z' < k + 1 for some k e Z; 

(2) w' = w e C\Z or I < z' < I + 1 for some I G Z. 

We also want the series J2\ -Piv(A) to converge, and this condition is equivalent 
to the additional inequality 

(3) z + z' + w + w' > -1. 

Under these conditions the choice of cat makes Pjv into a probability distribution. 

Theorem 1.3 [012]. Let z,z',w,w' satisfy the conditions (l)-(3) above. Then 
there exists a character x = X^^'^ '^'^ ^ of U (oo) such that 



x\u{N) = 5^ Pn{X)x^ 



with Pat (A) given by (1.8). 



In order to describe the spectral measures for x^^'^ '^'^ we need to switch to 
a different representation for the A's. First, we describe the measures Pn in a 
different way. 

Consider the lattice 

^(AT) ^ f ^' ^ is odd, 

\ Z + ^, AT is even, 

and divide it into two parts 



3e 



W _ ( N-1 



.n ={■ 



"out 



= {..., 



N+3 
2 ' 



N-3 
2 • 

N+1 



(TV) 

out 7 



N-3 TV- 



}u{^,™,...}. 



l-^out I — 



Let us associate to every highest weight A = Ai>--->Aiva finite point 
configuration X(X) C X^^'^ as follows: 

where p's and g's are the Probenius coordinates of the positive and negative parts 
of A as explained above. Note that A can be reconstructed if we know ^(A). 

The probability measure P/v(A) makes these point configurations random, and, 
according to the usual terminology [DVJ], we obtain a random point process. We 
will denote this process by Vn- 

Introduce a matrix L^^^ on X*-^-* x X^^^ which in block form corresponding to 
the splitting X^^^ = X'^^^ U X^^^ is given by 



A^^^ 
-(^(^))* 



where A^^'' is a matrix on Xout x Xi 



A^^'Ka^b) 



a — b 



aexi^l bex[^\ 



and f{x) was introduced in (1.8). 
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(r(-x+-±i)r(x+-±i))^ 

T{x + ^) \ 



Proposition 1.4 [B05]. For any highest weight A = (Ai > • • • > A^r) 

det L^vf\\ 
Pn{X) - ^ ^ 



det(l + L(^)) ' 

where L^x(X) denotes the finite submatrix of L^^^ on X{X) x X{X). Moreover, if a 
finite point configuration X C X^^"^ is not of the form X = X(A) for some highest 
weight X, then detL^^ = 0. 

Proposition 1.4 implies that Vn is a determinantal point process (see [So], [BOO, 
Appendix], [B05] for a general discussion of such processes). In particular, this 
implies the following claim. 

Corollary 1.5 [B05]. The matrix L^^^ defines a finite rank (and hence trace class) 
operator in £'^{di^^^). The correlation functions 

p^^\xi,...,Xk) = Piv{A| CX(A)} 

of the process Vn have the determinantal form 

) (xi, . . . , Xfc) = det[K(^) (x,, x,)]f,,=i, /c = 1, 2, . . . , 

where K^^\x,y) is the matrix of the operator K^^^ = L(^)/(1 + L(^)) m£2(xW). 
Explicit formulas for K^^^ can be found in [B05]. 

Now we will describe the limit situation as A?" — > oo. Define the continuous phase 
space 

3£(oo) ^ l^lj 

and divide it into two parts 

3C — ^in LI ^out; 
^in=(-|,|), Xout = (-00, -^) U (|,+Oo) . 

To each point u) E 0,° we associate a point configuration in X as follows: 

u) = {a'^ , P~^; a~ , (3~) 

^ X{u) = {at + 1} U (I - /?+} U {-aj - 1} U , 

where we omit possible zeros in a"*", a~ , (3^ , and possible ones in /?~. 

Let us denote by P = p(^'^ ) the spectral measure for the character x^^'^ '^'^ ^ 
given by Theorem 1.3, and let P" be its push-forward to Q°. Then using the above 
correspondence between points in 0° and point configurations, P° can be inter- 
preted as a measure on the space of locally finite point configurations in X, that is, 
as a point process. We will denote this process by V. 

Since the measures Pn converge to the spectral measure P° as A" — > oo (Theorem 

1.2), we should expect the correlations functions p^^^ to converge to the correlation 
functions of 7^ as A ^ cxo. 

For any x e X we will denote by xjv the point of the lattice X^-^^ which is closest 
to xN. 
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Theorem 1.6 [B05]. The correlation functions 
Pkixi, ...,Xk) 

P°{uj\Xiijj) intersects each interval ixi, Xi + Axj), z = 1, . . . , A;} 
= lim — ■ — — ^ 

Axi,..., Aajfe^+O i\Xi • ■ • Axf. 

of the process V have determinantal form 

pk{xi,...,Xk) = det[K{xi,Xj)]lj^-^, k = l,2,..., 

where K{x,y) is a kernel on X which is the scaling limit of the kernels K^^\x,y) 
introduced above: 

K{x,y)= lim iV-KW(a;jv,yjv), x,yeX. (1.9) 

JV— >oo 

The kernel K(x,y) is called the continuous 2-P1 kernel axid is precisely the kernel 
in (0.1) for x,y > |. Explicit formulas for K{x, y) can be found in the next section. 
This kernel is a real-analytic function of the parameters {z, z' ,w,w'). We will use 
the same notation for its natural analytic continuation. 

It is worth noting that the correlation functions pk{xi, . . . ,Xk) determine the 
process V uniquely. 

It is a well-known elementary observation that the probability that a determi- 
nantal point process with a correlation kernel /C does not have particles in a given 
part J of the phase space is equal to the Fredholm determinant det(l — /C| j), see, 
e.g., [So], [TWl].i 

In what follows we study determinants of the form det(l — K\j) where K is the 
continuous 2F1 kernel and J is a union of finitely many (possibly infinite) intervals. 

2. Continuous 2F1 kernel. Setting of the problem 

Following [B05] we consider the continuous 2F1 kernel with parameters satisfy- 
ing the conditions (1) -(3) of §1. 

To avoid unnecessary complications (poles in certain formulas below), we exclude 
the set where z+z'+w+w' = from our consideration. Most of the results, however, 
can be extended to this set by analytic continuation in one of the parameters. 

Recall that in §1 we introduced the space 

X = R\{±^} 

and divided it into two parts 

3C .^out LJ .^in, 
^out = (-00, -|) U (|,-hOo) , Xin=(-|,|). 



^If the correlation kernel is self-adjoint and this probability is nonzero then the integral operator 
defined by the kernel K. is of trace class and the determinant is well-defined, see [So, Theorem 4] . 
For kernels which are not self-adjoint, the existence of the determinant, generally speaking, needs 
to be justified, see, e.g., the end of Section 2 below. 
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Introduce the functions 



V'out(a^) = 



V'out : ^out 1^+, i^in '■ ^ir 



X > ^ , 



-z—z I 

1 ^ 9 ) 



2 < ^ < 2 ' 



^, ,, sin(7r2;) sin(7r2;') ^, ,, suvIttw) suvIttw') 

C{Z,Z) = 2 , C{W,W) = 2 , 



Note that C{z, z') > and C{w,w') > 0, so that i(^out{x) and 'i/'in(^) are positive. 

We now define the 2-P1 kernel on X. It is convenient to write it in block form 
corresponding to the splitting X = Xout U Xin: 



K 



-^in,out -f^in,in 



We set 



t N n -^out {x) Sout (y) - Sout (x) -Rout (y) 

x-y 

Rout{x)Rin{y) - Sout{x)Sin{y) 



out, in 



{X,y) = Vi^ont{x)i^in{y) 



Kin,out{x,y) = VVin(a^)V'out(y) 



X - y 

Rm{x)Rout{y) - Sin{x)Sout{y) 

X-y 

( X / , / X , /X R\xx[x)S;^{y) - ^in(a;)it;in(y) 

X y 



where 



-Rout(a^) 



-5out(a:) = r 



X + 



X 



z + w\ z' + w' 
z + z' + w + w' 



X 



z + w + 1, z + w' + 1, z' + w + 1, z' + w' + 1 
z + z' + w + w' + l, z + z' + w + w' + 2 



X 



X 2 2 



1 \ ™ 

a; + 2 



2 + ty' + 1, 2' + ty' + 1 

^ + ^' + w + «;' + 2 



— X 



„ , , Sm TTZ ^ 

i?in(a;) = r 



TT 



X I - + a; 



SlUTT^; 



z' — z, z + w + 1, z + w' + l 
z + w + z' + w' + l 



1 



— X 



z + w' + l, —z' — w 
z- z' + 1 



— X 



TT 



X I - + a: 



z - z' , z' + w + 1, z' + w' + 1 
z + w + z' + w' + l 



1 

2"" 



2F1 



z' + w' + 1, —z — w 
z' - z + 1 
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sm7r2; 
Sin{x) = r 



TT 



z' — Z, Z + z' + W + w' 
z' + w, z' + w' 



X 



+ X 



— X 



z + w', —z' — w + 1 
z- z' + 1 



— X 



Sm TTZ 
TT 



z — z' , z + z' + w + w' 
z + w, z + w' 



X 



1 

2+^ 



z' + w\ —z — + 1 
z' -z^\ 



1 



a, 6 
c 



is the Gauss hypergeometric function, see, e.g., [Er, Ch. 2], and 



mm 



Here 2F1 ' x 

the notation F , means ^ , 

r(c)r(rf) 

Note that for 2 = 2;', the functions i?in and 5'in are, formally speaking, not defined 
because of the presence of factors V{z — z') and V[z' — z). However, the formulas 
have a well-defined limit as 2; ^ 2;', because the second summands in the formulas 
for i?in and 5'in are equal to the first summands with z and z' interchanged. 

In the sequel we will need to know certain analytic properties of the 2F\ kernel. 
We discuss these properties below. 

2.1. Smoothness. K{x,y) is a real-analytic function in 2 variables defined on 
X X X. Its values on the diagonal are determined by the L'Hopital rule: 

V'out(a:)(i?out(3^)'S'out(a^) - S'o^^{x)Rontix)), X e Xout, 
'll;inix){R{Jx)Sin{x) - S{^{x)Rin{x)), X G Xin- 

2.2 Symmetries of i?out, 'S'out, ^in, Sin. AU four functions i^out, 'S'out, Rin, Sin 
are invariant with respect to the transpositions z z' and w w' . This follows 
easily from the above formulas and the identities 



K{x, x) 



2F1 
Since 



a, b 
c 



c 



^{i-cr-^-^F^ 



c — a, c — b 
c 



c 



iFi 



a, b 
c 



c 



— 2FI 



6, a 
c 



c 



2-^1 



a, b 


c 


— 2FI 


a, b 


c 


c 






c 





where the bar means complex conjugation, and the parameters {z,z'), as well as 
{w, w'), are either real or complex conjugate, the functions i?out) -S'out and Rm, Sin 
take real values on Xout and Xin, respectively. 

Further, let us denote by C the following change of the parameters and indepen- 
dent variable: {z, z', w, w', x) < — > {w, w', z, z' , —x). Then 

C(V'out) = V'out, C(V'in) = V'in, 
C(-Rout) = -Rout) C(jS'out) = "-S'out) C{Rin) = -Rin) C(jS'in) = ~5'in. 

For ipout and i/jin the claim is obvious from the definition. For Rout and ^out the 
symmetry relation follows from the identity 



iFi 



a, b 


c 


c 





(i-C)-Vi 



a, c — b 


C 1 


c 


C-iJ 



-(i-C)-Vi 



c — a, b 


C 1 


c 


c-ij 



For Rin and Sin the symmetry is a corollary of the symmetries of i/jin, -Rout) -Sout) 
and the branching relation (2.1) below. 
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2.3. Symmetries of the kernel. Since the functions -Rout; 'S'out) Rim Sin take 
real values, the kernel K(x, y) is real. Moreover, from the explicit formulas for the 
kernel it follows that 

-f^out,out(a^,y) = Kout,out{y,x), Kin,in{x,y) = Kin,in{y,x), 
Kin,out{x,y) = -Kout,in{y,x). 

This means that the kernel K{x,y) is (formally) symmetric with respect to the 
indefinite metric id©(— id) on L^(X, dx) = L^(3£out) dx) © L^(Xin, dx). 

2.4. Branching of analytic continuations. The formulas for .Rout; Sout: 

Sin above provide analytic continuations of these functions. We can view -Rout and 
S'out as functions which are analytic and single- valued on C\Xin, and i?in and Si^ as 
functions which are analytic and single- valued on C \ 3£out- (Recall that the Gauss 
hypergeometric function can be viewed as an analytic and single valued function 
on C\ [l,+oo).) 

For a function F{() defined on C\M we will denote by and F~ its boundary 
values: 

F+{x) = F{x + iO), F-{x) = F{x-iO). 
We will show below that 



1 Q- — Q+ 

^out ^out p 

on Xin — : — -Kir 

Win 27rz 



1 p— p+ 

-'''out -'''out c 

- — Oil 



on j£, 



1 



out 



i^in 
1 



27rz 



V'out 27rz ' Vout 27rz 

We will use the following formula for the analytic continuation of the Gauss hyper- 
geometric function, see [Er, 2.1.4(17)], 



'out 



(2.1) 
(2.2) 



iFi 



a, b 


c 




c 





T{b-a)T{c) 
T{b)T{c-a) 
T{a - b)T{c 



(-C)-"2Fi 



o, 1 — c + a 
1-b + a 



-C)-'2i^l 



6, 1-c + b 
1-a + b 



(2.3) 



T{a)T{c-b) 

This formula is vahd if 6 - a ^ Z, c ^ {0, -1, -2, . . . }, and C ^ 

Both of the formulas in (2.1) are direct consequences of (2.3) and the trivial 
relation 

(C)" - (C)+ sin(7r«) 



on 



27Ti TT 

To verify the first formula of (2.2), we use the relation (2.3) for both hypergeo- 
metric functions in the definition of Sin- Thus, we get 4 summands in total. After 
computing the jump {S^ — S^)/27ri, the second and the fourth summands cancel 
out. As for the first and the third summands, they produce exactly V'out-Rout) which 
can be seen from the identities 

r(s)r(i -s) = — see. 



sin(7r(z -|- w)) sin(7r(2: + w')) 



+ 



sm[ns) 

shi{'K{z' + w)) sin(7r(z' -|- w')) 



= 1. 



sin(7r(2; + z' + w + w')) sin(7r(2; — z')) sin(7r(2; + z' + w + w')) sin(7r(2;' — z)) 

The second part of (2.2) is proved similarly. 

The restriction 6 — a ^ Z for (2.3) in our situation means that our proof works 
when z' ^ z. For z' = z the result is obtained by the limit transition z' ^ z in 
(2.1) and (2.2). 
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2.5. Differential equations (due to G. Olshanski). We use Riemann's nota- 
tion 

'ti t2 h 

P \ a b c C 
a' b' c' 



to denote the two-dimensional space of solutions to the second order Fuchs' equa- 
tion with singular points ti,t2,t3 and exponents a, a'; b,b'; c,c', see, e.g., [Er, 2.6]. 
If a — a' ^ Z then this means that about ti , there are two solutions of the form 

(C ~ ^i)" ^ holomorphic function}, {( — ti)°' x {a holomorphic function}. 

If a = a' then the basis of the space of solutions near ti has the form 

{C—ti)°'x{a holomorphic function}, \n{(—ti){C—ti)°'x{a holomorphic function}. 

The holomorphic functions above must take nonzero values at ti. For t2 and the 
picture is similar. 

We always have a + a' + b + b' + c + c' = 1. 

The Gauss hypergeometric function 2F1 ^ 



c 



belongs to the space 



00 1 
P I a C 

1 — c b c — a — b 

and, since it is holomorphic around the origin, it corresponds to the exponent at 
the origin. 

Riemann showed (see [Er, 2.6.1]) that 

P I a b c(^\=Pla + >c b — k — ji c + n Q 
\a' b' c' ) \a' + K b' - K- n c' + jj, 

(2.4) 

where if = 00 then the factor C — tn should be replaced by 1, and 

h t2 h \ I Si S2 S3 

P \ a b c C] =P \ a b c r] 
a' b' c' \a' b' c' 



C-t2j \C-t2 



where 



_ AC + B _ Atn + B _-, . „ 



A, B,C,De C, AD-CB^ 0. 
Using these facts, we immediately see that (denote & = z + z' + w + w' ^ 0) 



Rout{x) eP \ w z x \ . (2.5) 
w' 1-e z' 
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Moreover, i?out is the only element of this space which corresponds to the exponent 
at the infinity and has asymptotics 1 there. 
Similarly, 

/-I oo 1 \ 
^out(^) eP\ w 1 z x], (2.6) 
\ w' -6 z' J 

and this is the only element of this space, up to a multiplicative constant, with the 
asymptotics const ■x~^ at infinity. 
Hence, by (2.1) and (2.4) we get 

I ~\ ^ \ \ ( ~\ ^ \ \ 

Rin{x) eP{ -W' -Z' x \ , Sin{x) E P \ -w' 1 -z' X \ . (2.7) 

\-w 1 + & -z J \-w & -z J 

2.6. Asymptotics at singular points. The results of the previous subsection, 
see (2.5)- (2.7), imply that near ( = ^, if z ^ z' then 

i?out(c) = ci (c - 1)^ (1 + o (c - i)) + C2 (c - '^y' (1 + o (c - D) , 

5out(C)=C3 (C-|)'(l + 0(C-i))+C4 (C-l)'' (l + 0(C-i)), 
i?in(C) = C5 (C - i) (1 + O (C - i)) + C6 (C - i) (1 + O (C - i)) , 

5in(C) = cr (C - i)"^ (1 + O (C - I)) + cs (C - 1)"^' (1 + O (C - I)) . 

Here and below we denote constants by the letters Cj, i = 1, 2, 

If 2; = 2;', we have 

i?out(C) = ci (C-i)^(l + 0(C-i))+C2ln(C-i) (C-i)^(l + 0(C-i)), 
-5out(C) = C3 (C-i)'(l + 0(C-|))+C4ln(C-i) (C-i)^(l + 0(C-|)), 

i?in(C) = C5(C-^)"' (l + 0(C-i))+C6ln(C-i) (C-l)"^ (l + 0(C-i)), 

5i„(C) = C7 (C-^)"^ (l + 0(C-i))+C8ln(C-i) (C-i)"^ (l + 0(C^i)). 

Similar formulas hold near C = ~| with the parameters {z,z') substituted by 
{w, w'). 

Since the Gauss hypergeometric function is holomorphic around the origin, the 
definitions of i?out and /Sout imply that as C — > 00, 

i?out(C) = 1 + 0(C-^), 5out(C) = ci c-'(i + 0{C')), 

Kut(C) = C2 C-'(i + 0{C')). ^Lt(C) = C3 C-'(i + 0{C')). (2.8) 

<ut(c) = C4c-'(i +o(c-^)), <s;:ut(c) = C5c-'(i +o(c-^)). 

As for i?in and ySm, the results of the previous subsection, see (2.7), imply that, as 

C ^ 00, 

RUC) = ci(i + o(r^)) + C2c'-^{i + o(r^)), 

C2C-ni + 0(C-^)) + C3C-®(1 + O(C-i)), 6 ^ 1 (2.9) 

C4C-'(1 + 0(C-^)) + C5 ln(C)C-'(l + 0(C-^)), 6 = 1. 



^in(C) 
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We will need the exact value of ci in (2.9) later on. In fact, ci = 1, and 



i?in(C) = l + 0(C-^) + C»(C-^-®), C^oo. (2.10) 

To prove this we do a similar calculation as in the verification of (2.2) above. That 
is, we use the relation (2.3) for both hypergeometric functions in the definition of 
i?in. Then out of the four summands that arise, the first and the third summands 
give contributions of order and higher, while the second and the fourth 

ones produce a function in the variable ((^+ |)~^ holomorphic near the origin with 
constant coefficient 1. 

Now we are ready to formulate the problem. Let 

J = (ai, 02) U (as, 04) U • • • U (a2m-i, a2m) C R, 

(2.11) 

—00 < ai < a2 < • • • < a2m < +00, 

be a union of disjoint (possibly infinite) intervals inside the real line such that the 
closure of J does not contain the points ±i. Denote by K"^ the restriction of the 
continuous 2F1 kernel K{x, y) introduced above to J. Our primary goal is to study 
the Fredholm determinant det(l — K^). 

In the last part of this section we justify the existence of this determinant. 

Denote 

J out — </ n Xout? J'nn t/ n Xiji, 

■'^ out, out ^ MoutX Jout' -'^illjin ^ MinX Jin) 

Ty-J Ty-\ jy-J jy-t 

-'^■OUt,!!! I Jout X Jin ) -'*-in,OUt I Jin X Jouf 

Proposition 2.7. The kernels -f^out,out(^5 2/) ^'^^ -f^in,m(^) 2/) define positive trace 
class operators in L'^{Jout,dx) and L^{Jin,dx), respectively. 

Proof. 2.1 and 2.3 above imply that the kernels K^^^^^^^{x,y) and Kf^^^{x,y) are 
smooth, real-valued, and symmetric. Moreover, the principal minors of these kernels 
are always nonnegative, because the kernel K was obtained as a limit of matrices 
with nonnegative principal minors, see §1. Thus, it remains to prove that the 
integrals 

/ -^OTt,out(^5 / -f^in,in(^) ^)^^ 

•^Jout •'Jin 

converge. For the second integral the claim is obvious since Jin C (— |, |), and the 
integrand is bounded on Jin. For the first integral we need to control the behavior 
of the integrand near infinity (if Jout is not bounded). Since V'out(2;) = 0(a;~®) as 
a; — > 00, by 2.1 and (2.8) we see that 

K{x, x) = 0(a;-2-®), x ^ 00. 

As 6 > —1, the integral converges. □ 

We will assume that K^^^^^^{x,y) = and K■^J^^^^^{x,y) = if (x^y) does not 
belong to the domain of definition of the corresponding kernel (Jout x Jin for the 
first kernel and Jin x Jout for the second one). 
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Proposition 2.8. The kernel Ko(x,y) = -f^out,in(^; 2/) + -^d,out(^' 2/) defines a 
trace-class operator in L'^{J,dx). 

,2 

Proof. Consider the operator acting respectively on 

(i) Co~W; (ii) Co~(J); (iii) Co~(M\J). 



In all three cases the operator is essentially self-adjoint, giving rise to the positive 



self-adjoint operators i7, Hj, and H^^-j in L2(M), L2(J), L2(R\ J), respectively. It 



is well known (see e.g. [RS, Theorem XI.21]) that the operator T — [1 + x'^)~^{l + 
H)~^ is trace class in L^(R). A direct proof can be given as follows. Let p denote the 
(self-adjoint) closure of —i-^ acting on Cq°; then H = p^. Commuting (1 — ix)~^ 
and ip)~^ in the representation 

T = (1 + ix)~^{l - ix)~^{l + ip)~^{l - ip)~^, 

we obtain the formula 



T = ((1 + ix)~^{l + ip)~^) ((1 - ix)-^{l - ip)~^) 
+(1 + ixy^il + zp)~^(l - ix)-^[x,p] (1 - ix)-^{l + ip)~^{l - ip)~^ . 

But a simple computation shows that (1 + ix)~^{l + ip)~^ has kernel 

(1 + ix)-^xo{y - x)e-''-y , 

where xo denotes the characteristic function of (0, 00), and as 

(1 + x^)~'^e''~ydxdy < 00, 



(2.12) 



/ ( 

J y>x 



it follows that {1 + ix) ^{1 + ip) ^ is Hilbert-Schmidt. The same is true for (1 — 
ix)~^{l±ip)~^ and as [a;,p] = z, the trace class property for T follows immediately 
from (2.12). 

For / e L2(M), set 

^ = ((1 + H)-^ - (1 + {Hj © if^v))"') /• 

The function g solves ( — + = in the following weak sense: if G Co°(]R \ 
{ai, . . . , a2m})i then {{~'^ + gdx = 0. It follows that in each component 

of M \ {ai, . . . , a2m}, (7 is a linear combination of the functions and e~^, and 
hence the operator (l + i7)~^ — (1 + {Hj © Hg^^-j))~^ is of finite rank. As T is trace 
class, it follows, in particular, that (1 + a;^)~^(l + Hj)~^ is trace class in LF'{J). 
Observe that the kernel -fCout,in(a^, J/) has the form 

F^{x)G^{y) + F2{x)G2{y) 



x-y 
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for suitable functions Fi,Gj. For x E Xouti V G ^in, set 



i^l(x,2/)=i^out,in(x,2/)- I ( ^Gi(|/) + ^G2(2/) 



X X 



+ [^yG,{y)+'^yG2{y)]\v{x). 

Here V{x) is a smooth function on R which is zero for |a;| < L = max{|aj| : |aj| < 
oo}, and V{x) = 1 for > L + 1. 
FinaUy, for x e Xout) y £ ^in set 

K2{x,y) ^ Ki{x,y) - ^ Xai{x)Ki{ai,y), 

|0i|<00 

where the sum is taken over all the finite endpoints of J. Here Xoi [x] is a smooth 
function compactly supported in J, which equals 1 in a neighborhood of a^, and 
which vanishes at aj for j ^ i. Clearly K2{x,y) = for x G dJ, which implies 
that K2{- ,y) e dom iJj for all y e 3£out- Using the decay conditions (2.8) (each 
differentiation with respect to x gives an extra power of decay) it follows that 
(1 + Hj){l + x'^)K2{x, y) gives rise to a bounded operator on L^{J), and hence 

K2 = ((1 + xY\i + Hj)-') ((1 + Hj){l + x^)K2) 

is trace class. But clearly K2 is a finite rank perturbation of -ftTout,!!!- A similar 
computation is true for Kin,out5 ^-nd we conclude that Kq is trace class on L'^{J). □ 



Proposition 2.7 and Proposition 2.8 prove that the operator 



out, out out,m 



^*^in,out " 



is trace class. This shows that the determinant det(l — K ) is well-defined. 

3. The resolvent kernel and the 
corresponding riemann-hllbert problem. 

Starting from this point we assume that the reader is familiar with the material 
in the Appendix. 

As was explained in §1, see Theorem 1.6 et seq., the 2F1 kernel ii' is a limit of 
certain discrete kernels which we denoted as K^^\ Moreover, these discrete kernels 
have rather simple resolvent kernels L^^'^ = K^^^/{1 — K^^^), see Corollary 1.5. 
The kernels L^^'^ are integrahle, and, thus, the kernels K^^^ can be found through 
solving (discrete) Riemann-Hilbert problems, see [Bor2]. 

Our first observation is that the kernel L^^^ admits a scaling limit as AT —> 00. 
Recall that for a; e X we denote by a;jv the point of the lattice X^^^ which is closest 
to xN. 

The proof of the following Proposition is straightforward. 
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Proposition 3.1 [B05]. The limit 



L{x,y)= lim N ■ L^^\xN,yN) , x,ye3e, 

AT— >oo 

exists. In the block form corresponding to the splitting X — Xout U ^in, the kernel 
L{x,y) has the following representation: 



L = 





-A' 



A 




where A is a kernel on Xout x ^in of the form 



A{x,y) = 



^/'^pout{x)'^pin{y) 



x-y 

where the functions V'out o-i^d '4>\n were introduced at the beginning of%2. 

Now an obvious conjecture would be that K = L(l + L)~^, and K can be 
obtained through a solution of the corresponding Riemann-Hilbcrt problem. Both 
claims are true, but under certain restrictions on the set of parameters {z, z' , w, w'). 
We begin by showing how to obtain K from a RHP. 

Observe that the formulas for the 2F1 kernel given in §2 are identical to (A. 2) 
in the Appendix with 



m 



mil 


mi2 




Rout 


m2i 






'S'out -^in 



hi = VV'out, hii = -x/V^. (3.1) 



In particular, this means that the 2F1 kernel is integrable. Clearly, the matrix- 
valued function m is holomorphic in C \ M, and as we will see, detm(C) = 1 (see 
proof of Proposition 3.3 below). 



Proposition 3.2. The matrix m solves the Riemann-Hilbert problem (X, v) with 

X G Xout; 



v{x) 



1 2'Ki'lpoutix) 

1 

1 
27ri'0in(a;) 1 



(3.2) 



If in addition z + z' + w + w' > 0, then m{() ~ / as C ^ oo- 

Proof. The jump condition m_|_ = ni-V is equivalent to (2.1), (2.2). The asymptotic 
relation m ~ 1 at infinity follows from (2.8), (2.9), (2.10). □ 

Note that the condition z + z' +w + w' > is only needed to guarantee the decay 
of mi2 = —Sin at infinity, see (2.9). 

Now we investigate the nature of the singularities of m near the points ±| of 
discontinuity of the jump matrix v. We will need this information further on. 

Introduce the matrix 



c(C) 



z + z' 



(C-i)^(C + i) 





w-\-w 





zA-z' w-\-w' 



(3.3) 
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Observe that C is holomorphic in C \ (—00, |]. Furthermore, on (—00, |) 



C_{x){C^{x))-^ ^ I 



^—i-K(z + z') Q 
^-iTT{z + z' +W + w') Q 

^in{z+z' +w+w') 







a; e (-00, -|) , 

(3.4) 



is clearly a piecewise constant matrix. 

Proposition 3.3. (i) Assume that z 7^ z' . Then near the ■point C = | 



(C-l)^ 

(c-l) 

z-z' 

(C-l)^ 







(c-l) 



C/2, »C < 



/or some nondegenerate constant matrices Ui and U2 and locally holomorphic func- 
tion Hi /2(C) such that Hi/2{^) is also nondegenerate. 
(a) Assume z = z' . Then near the point C = | 



m(C)C-(C) 




V2, ^c<o 



for some nondegenerate constant matrices V\ and V2 and locally holomorphic func- 
tion -ffi/2(C) such that -ffi/2(|) is also nondegenerate. 

Proof. Let us assume first that z ^ z' . Define a new matrix m(C) as follows 



m(C) = 



■ 1 27rz C(^, z') ■ 
1 



(The constants C{z, z') and C{w,w') were defined at the beginning of §2.) 

By (3.2) we see that the jump matrix v for m locally near the point | has the 
form 



V 



1 -2mC{z,z') 
1 



1 -2TiiC{z,z') 
1 



-iir{z-\-z') 
2TTi 







-1 



^177(2+2') 



X > 25 



Note that this matrix is piecewise constant. 

An easy computation shows that for a certain nondegenerate matrix t/, 



1 -2mC{z,z') 
1 



-i7r(z+2') 

27ri 





3i7r(z;+z;') 



gi7r(z-2;') Q 

gi7r(2'-z) 
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This implies that 



TOo(C) 



(c-l)- 







(C-l) 



u 



locally solves the RHP with the jump matrix v. 

Our conditions on the parameters {z, z', w, w') imply that \^{z — z')\ < 1. Then 
the asymptotic formulas of subsection 2.6 imply that m is locally square integrable 
near C = |) so are mo and friQ^, as follows from the formula above. Since m and 
fho locally solve the same RHP, we obtain that mniQ^ has no jump on R near C = 
and it is locally integrable as a product of two locally square integrable functions. 
Hence, this ratio is a locally holomorphic function. We denote this holomorphic 
function by Hi /2(C) ^ ^ind set 



Ui = U, U2 = U 



1 -27riC{z,z') 
1 



As v{x) in (3.2) has determinant 1, it follows that detm_|_(a;) = dctm_(a;). Also, 
as above, detm(C) = detm(C) is locally integrable. Thus, detm(C) is entire. If 
z + z' + w + w' > 0, then as noted in Proposition 3.2, detm(C) — >^ 1 as C — 00, 
and hence, by Liouville's theorem, detm{Q = 1. Analytic continuation in the 
parameters z,z' ,w,w' ensures that the same is true for all (allowable) values of 
the parameters. The fact that i?i/2(|) is invertible now follows from the fact that 
detm(C) = detC(C) = 1, and detC/i, det C/2 are nonzero. The proof of (i) is 
complete. 

Assume now that z = z' . Then there exists a nondegenerate matrix V such that 



1 -2TTiC{z,z') 
1 



2m 







V- 



1 1 

1 



and the local solution of the RHP with the jump matrix v has the form 



™o(C) 



1 In(C-i) 
1 



V. 



Repeating word-for-word the argument above we get (ii) with 



Vi = V, V2 = V 



1 -2TTi C{z, z') 
1 



□ 



Similarly to Proposition 3.3 we have 
Proposition 3.4. (%) Assume that w ^ w' . Then near the point C = — | 



H -1/2(0 



H -1/2(0 



(c + l) 
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(C+l) 


(c+l) 



w —w 



Ui, 9C>o, 



U2, QC<o 



for some nondegenerate constant matrices Ui and U2 and locally holomorphic func- 
tion H_i/2{C) such that H_i/2{~^) is also nondegenerate. 
(a) Assume w = w' . Then near the point C = ~i 



m(C)C-^(C) 



^-1/2(0 



1 ln(C+i) 

1 

1 ln(C + i) 
1 



V2, ^c<o 



for some nondegenerate constant matrices Vi and V2 ond locally holomorphic func- 
tion i/_i/2(C) such that H_i/2{—^) is also nondegenerate. 

We now return to the question raised after Proposition 3.1, whether the kernel 
L{x,y) provides a resolvent operator for the 2F1 kernel K. The reason why we 
cannot immediately apply the general theory of the Appendix in this case is that 
the functions fi, gi (or hi — \/VVut7 — VV'm) in the notation of the Appendix 
are not bounded on the contour as required by (A.l). We proceed rather by direct 
calculation. 

First of all, we determine when the operator L is bounded. 

Proposition 3.5. The kernel L{x,y) defines a bounded operator in L'^{X,dx) if 
and only if \z + z'\ < 1, \w -\- w'\ < 1. 

Proof. It suffices to consider the operator A : L^(j£in, dx) — > L^(j£out) dx) with the 
kernel A{x, y) = •\/Vout(a^)'0in(2/)/ (x-y). 

If \z + z'\ > 1, say, z + z' > 1, then the restriction of A{x,y) to (|, 1) x (— i,0) 
is a positive function in 2 variables bounded from below by 



= WC{z,z')\x 



z + z' 



\x + 



\y + i\ 



w-\-w 



This kernel has {x — h] 



behavior near x — ^. Thus, A is unbounded. 



Similarly, we see that A is unbounded if z + z' < —1 or |w + w'| > 1. 

Now assume that |2; + 2;'| < 1, \w + w'\ < 1. Let x be the characteristic function 
of the set (—00, — | — e) U (| + e, +00) for some e > 0. Then the kernel x{x)A{x, y) 
defines a Hilbert-Schmidt (hence, bounded) operator on L'^{X,dx). Indeed, 



- X 



\x{x)A{x,y)\^dxdy 



+00 




{X - 



dx + 



—00 



V'out(a^) 



dx 



il^in{y)dy < 00. 



Hence, in order to prove that L is bounded, it is enough to show that for any 
compactly supported smooth functions / on Xout? supp /C [— ^ — e, |)U(|,| + e], 
and g on 3£in, 



^{x,y)g{x)f{y) 



< const 



2||^l|2- 



(3.5) 
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We will assume that / is supported on ( | , | + e] . The case when / is supported 
on [— i — e, — I) is handled similarly. Assume that g is supported on [0, |). Let us 
introduce the polar coordinates (r, 9) by 

x—\—rcos6^ |— 2/ = rsin6', < 6* < ^, < r < r(6'), 

for some r{9) < const < oo. Then the integral above takes the form 

" r(^) (cos^)-"^(l + rcos^)-^(sin^)^(l-rsin^)^ 



^0 



cos 6' + sin 6' 
xg {rcosO + |) / (| — rsin^) drdO. 

(3.6) 

Here r{9) is a uniformly bounded continuous function of 6. Clearly, the factors 
|a;+ i|-^^^^^T^ = (1 + rcos^)~^^^^^2^ and |y+ = (i — rcos^)^^^^^ are bounded 

on the domain of integration. Using the inequalities 



poo 

I g {r cose ^ \) f {\-r sine) dr 
Jo 

cos 9 + sine > 1, 



< (cos^sin^)-^||/||2||^||2, 



we see that the integral (3.6) is bounded by 

const / (cos^)-^^^(sin^)^^V^d^ • H/lbHiflb < const H/lblliflb- 
Jo 

If / is supported on (|, | + e] and g is supported on (— |, 0] then the denominator 
in A{x, y) is bounded away from zero, and A is bounded by simple estimates. This 
completes the proof of (3.5) in the case that / is supported on (i, | + e] and g is 
supported on (— |, i). □ 

Since L* = — L, we know that if L is bounded then (1 + L) is invertible. It 
seems very plausible that whenever the operator L is bounded, the relation K — 
L{1 + L)~^ should hold. We are able to prove this under the additional restriction 
z + z' + w + w' > {). 

Proposition 3.6. Assume that z -\- z' -\- w -\- w' > \z -\- z'\ < 1, \w -\- w'\ < 1. 
Then K = L{1 + L)-K 

Proof. Since L is bounded and L = —L*, L has a pure imaginary spectrum, and 
1 + L is invertible. Hence, it is enough to show that K + KL = L. The restrictions 
on the parameters and the asymptotics of the functions i?out, 'S'out, -Rin, 'S'in, from 
subsection 2.6 imply that the relation (2.1) and (2.2) can be rewritten in the integral 
form: 



x-y Jx^„, x-y 



(3.7) 
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The I's on the right-hand side appear because -Rout(C) ~ 1 ^^d -Rin(C) ~ 1 as 
C — > oo. The restriction z + z' + w + w' > is needed to ensure the convergence of 
the first integral at infinity. Indeed, '4'ont{.^)Ro-at{.^) ~ x~^~^ -w-w ^ _^ 
The identity 

K{x,y) + I L{x,a)K{a.,y)da = L{x,y) (3-8) 

for aU x,y e X foUows directly from the relations (3.7) (see [B02, Theorem 3.3] for 
a similar computation). On the other hand, by (2.8) we see that for any g e Cq°(X), 



G{a) = / K{a,y)g{y)dy = Kg{a) 
Jx 

lies in L'^{X,da). Integrating (3.8) against g{y), we see that (1 + L)G — Lg and 
hence Kg = {l + L)~^Lg in L^(3£). It follows that K extends to a bounded operator 
{1 + L)~^ L = L{1 + L)~^ in L^(X). Conversely, we see that the bounded operator 
L(l + L)~^ has a kernel action given by the 2F1 kernel K{x, y). □ 



Proposition 3.6 has the following corollary which will be important for us later 



on. 



Corollary 3.7. Assume that z + z' + w + w' > 0, \z + z'\ < 1, \w + w'\ < 1. Then, 

in the notation of ^2, the operator 1 — K"^ is invertihle. 

Proof. In the block form corresponding to the splitting J = Jout U Jin, the operator 
1 — K"^ has the form 



1 — K-^ 

out, out 

in, out 



out,in 



is invertible 



But it is easy to see that an operator written in the block form 
if a is invertible and {d — ca~^b) is invertible. Therefore, it is enough to prove that 

1 ~ -^out.out (1 ~ -^in,in) — -f^injOutll ~ -^out,out) -^out.in 

are invertible. 

Proposition 3.6 and the definition of the operator L imply that 

i^out,out = 1 - (1 + AA*)-\ = 1 - (1 + A*A)-\ 



Hence, -ftTout.out and -ftrin,in are positive operators which are strictly less then 1. 

and -ft^in,in- III particular, 1 — -K'out.out invertible. 
-(^ilout)*- Hence, 



Thus, same is true for K;^^^ 
Further, K^^^^-^ 



(1 -^in,in) -^in,out(-'- -^out.out) "^out.i 

is invertible. □ 



(1— JCin,in)+bounded positive operator 



Remark 3.8. It is plausible that the operator 1 — K"^ is invertible without any 
restrictions on the parameters (as opposed to the full operator 1—K which definitely 
ceases to be invertible if we remove the restrictions 1^ + z'| < 1 and |w + w'| < 1). 
However, we do not have a proof of this. In a similar but simpler situation of the 
Whittaker kernel we will prove the corresponding statement in §8.2 (see part (3) of 
Proposition 8.4 below). 
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4. System of linear differential 
equations with rational coefficients 



Our goal in this section is to show that the kernel of the (trace class and hence 
Hilbert-Schmidt) operator 

can be expressed through a solution of a system of linear differential equations 
with rational coefficients. This result will be crucial in our study of the Fredholm 
determinant det(l — K^) in the next section. 

In what follows wc assume that G = z ^ z' -\- w + w' > 

As noted at the beginning of §3, K is an integrable kernel: 

^ ) ^ F^{x)G,{y) + F2{x)G2{y) 
x-y 

Hence, K"^ is an integrable kernel. Since J is bounded away from the points ±|, it 
is easy to see that the functions F^, Gi (which are, in fact, the functions \/VWt-Rout: 
■s/VWt 'S'out 5 VV'in-Rin) VV'in'S'in rearranged in a certain way) belong to LP{J,dx) fl 
L°°{J,dx) for any p > 26"^ This follows from (2.8), (2.9). Set 



l-27riFiGi -27riFiG2 
-2TTi F2G1 1 - 27rz F2G2 



vj = I- 2ni FG^ -- 
Note that F\x)G{x) = Fi{x)Gi{x) + F2{x)G2{x) = 0. 

Proposition 4.1. Assume that the operator 1 — is invertible. Then there exists 
a solution mj of the normalized RHP (J, vj) such that the kernel of the operator 
= K^{1 - K^)-^ has the form 

Rj(x, y) = ^ 

x-y 

T = mj+F = mj-F, Q = mj+G — mj-G. 

The matrix mj is locally square integrable near the endpoints of J. 

Proof. See Proposition A. 2 and the succeeding comment. □ 

Concerning the invertibility of (1 — K"^), see Corollary 3.7 and Remark 3.8 above. 
Later on we will need the following property of the decay of mj at infinity. 

Proposition 4.2. As ( ^ 00, ( e C\R, we have m'j{C)mj^{C) = odCI"^)- 

Proof. We will give the proof for J = (s, +cxd), s > ^. The proof for general J is 
similar. 

Observe that detvj = 1. Then det mj has no jump on J. Since mj is square 
integrable near t, det mj is locally integrable. Moreover, det mj(C) ^ 1 as C ^ 00, 
because mj{Q I. Again by Liouville's theorem, det nij = 1, and mj^ is bounded 
near Q — 00. Therefore, it suffices to show that m'jiC,) — o(|C|~^). 

The proof of Proposition A. 2 given in [De] implies that for ^ e C \ R 

•+~mj+(t)F(t)G'(() , f+''mj.(t)F(t)G'((l^^ 



Js ^ C Js 
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t-C 



Therefore, 



mAQ--J^ (I^CP Js 



it - cy 



If > const I CI, then \t — (\ > const \(\. That is, the distance of the point C to the 
contour of integration is of order \(\. Since mj{t) is bounded and F{t)G^{t) decays 
at infinity as a positive power of t, we see that TOj(C) = odCl""*^)- 

If the point C, is closer to the real line and, say, < 0, we can deform the line 
of integration up to the line s + te*^, < 6* < ^. In other words, 

, _ mjjs + te'^)F{s + te'^)G\s + te'^) 

Here it is crucial that the vector-functions F and G (which are expressed in terms of 
the functions A/i/'out-Rout and VV'out'S'out) have analytic continuations in the sector 
< arg(C — s) < 9. Now the distance of the point ( to the contour of integration 
is again of order \(\, and the argument above again implies m'j{C) = odCl""*^)- If 
> 0, the proof is similar with the line of integration deformed down. □ 

We now describe a general procedure (cf. Steps 1, 2, and 3 in the Introduction) 
to convert RHP's with "complicated" jump matrices to RHP's with "simple" jump 
matrices. The procedure will be used again in Sections 8 and 9 to analyze a variety 
of other examples of integrable kernels. 

Lemma 4.3. Suppose S = S1US2, SinS2 = 0, is a decomposition of the oriented 
contour E C C into two disjoint parts. Suppose v is a function on E with values 
in Mat(/c,C). Suppose m, mi solve the RHP's (E,f), {Tii.v), respectively. Then 
if exists, 1112 ~ mim~^ solves the RHP (E2,f2) where V2 = mj^v~^m^^ = 
m-V~^mZ} ■ Conversely, ifm, m2 solve the RHP's (S, v) and (S2, ^2), respectively, 
then mi = m2m solves the RHP (Si,v). 

Proof. Direct calculation. □ 

Recall that, as noted at the beginning of §3, the formulas for the kernel K are 
identical to (A. 2) with m, /i/, hn given by (3.1). This, in particular, means that 



F = m±f, G = m^*g 



with 



r r ^ r ^ I V^VWtM) ^ ^ ^out, . , x . x / 0) xeX, 
fi{x) = g2{x) = \ ^ f2{x) = gi{x) - 



out) 



0, X e Xin, I y/lpin{x), X e Xin. 

Note that the matrix v in (3.2) has the form / + 27ri fg*. 

Lemma 4.4. // the matrix v in Lemma 4-3 has the form v = I + 27rifg* for 
(arbitrary) f and g with f^{x)g{x) = 0, then 



V2 ^I-2niFG\ 
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(4.1) 



where F = m^f = rri-f and G = m_^g = mj'g. 
Proof. Have 

V2 = m+ {I - 2771 fg^)m'^^ = / - 27rz (m+f) {m^^gY. □ 

Let E = X, El = X \ J, and E2 = J. Now mj solves the RHP (E2,f2) with 
V2 = vj = I — 2iiiFG^. But as noted above, F = m±f, G = m^*g, and so it 
foUows by Lemmas 4.3 and 4.4 that 

mi = mx\j = mjm 

satisfies the RHP (Ei,t;), where v = I + 2Tiifg^ as before. We think of V2 as 
the "comphcated" jump matrix and v as the "simple" jump matrix. The formula 
T^j = "^XXJ™""*^ shows that the analysis of the solution of the "complicated" RHP 
(J, vj) reduces to the analysis of the solutions of two "simple" RHP's {X\J, v) and 
{X,v). 

These two RHP's are "simple" for the following reason. Recall that in §3 we 
have introduced a matrix C{Q, see (3.3). 

Set M = 'mx\j C~^. This is a holomorphic function on C\R which has boundary 
values M± (x) on M. 

Lemma 4.5. The matrix-valued function M{() satisfies the jump relation M+ = 
M-V , where the jump matrix V has the form 



V{x) 



1 2TTiC{z,z')x{xy 
1 

. 2mx{x) e^'^^^+^'V 
Q-i^z+z' +w+w') 2TTiC{w,w')x{x) 

g ^iTT{z+z'+w+w') 



X > 



2' 



^ > X > „ , 



X < - 



2' 



where xi^) = Xx\j{x) is the characteristic function of the set X\J. 
Proof. On X \ J we have 

V = MZ^M+ = C-vC+^ = C-C+^ + 27Ti C-f {C^*g)\ 

and on J we have V = C-C^^. The jump relation (3.4) and explicit formulas for 
C, /, and g conclude the proof. □ 

The important fact about the jump matrix V is that it is piecewise constant. As 
discussed in the Introduction, this allows us to prove the following central claim. 
Recall that J is a union of m intervals with endpoints {aj}|^i, see (2.11). 

Theorem 4.6. The matrix M satisfies the differential equation 



M(C) 
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with some constant matrices A, B, and {Cj}^^i. If ai = — oo then Ci = 0, and if 
0'2m = +00 then C2m = 0. Other than that, all matrices A, B, {Cj}^^i are nonzero. 
Moreover, 

TrA = TrB = TrC. = 0, 



det^ = 



z — z 



detB = 



w — w 



, detC, = 0, 



for all j = 1, . . . , 2m, and 



2m 

^ + B + ^Cj = 



z + z' + w + w' 



1 

-1 



Proof. Since M satisfies the jump condition with a piecewise constant jump matrix 
V (Lemma 4.5), M' satisfies the jump condition with exactly the same jump matrix. 
Therefore, M'M"^ has no jump across X. (Note that 

det M = det mj det m(det C)"^ = 1, 

and hence exists.) 

Thus, we know that M'M~^ is a holomorphic function away from the points 
{±i} U {ai}?Z!\. We now investigate the behavior of M near these points. 

Near C — \^ f^j{C) is holomorphic, and the behavior of m{QC~^{C,) is described 
by Proposition 3.3. This implies, in the notation of Proposition 3.3, that for z ^ z' 



M'(C)M-i(C) = --Tmj(i)ifi/2(i) 

2 



2 






z'-z 



and for z = z' 



M'{OM-\0 = --^mj{l)H,/,{l) 
^ 2 



1 




^r/2(i)™7'(i)+o(i). 



Similarly, near C = — i we have the following: for w ^ w' 



M'{OM-\0 
1 



C+2 

and for w = w' 



2 






w' —w 



HZln (-i)m7i(-i)+0(l), 



1 




HZl,2 {-\)^~/ (-|)+0(1). 



As for the points {aj}^^^^ we will prove the following claim. 
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Lemma 4.7. In a neighborhood of any finite endpoint aj, j = 1, . . . , 2m, 



where hj{C) is holomorphic near aj, and Cj is a nonzero nilpotent matrix. 

Let us postpone the proof of this Lemma and proceed with the proof of Theorem 
4.6. Observe that if we set 



2 



1 




B= < 



(a Hy, (1) 

mj {-\)H_y, (-1) 

{-\) ^-1/2 {-\) 

then the function 

M\C)M-\0 







z = z , 



m.. 



1 


1 






w' —w 



w = w , 



A 



c-l 



+ 



B 



2m 



(4.2) 



is entire. At infinity we have, using mx\j = mjm, 

M'M~^ = m'jrrij^ + mjm'm~^mj^ + mjm(C~^)' Cm~^m'^^ . 

We know that m ^ I, rrij ~ /, m'jnij^ = o{\(\~^) (see Proposition 4.2), m'm~^ = 
odCl""*^) (which follows from (2.8) and differentiation of (2.9)), and by direct com- 
putation 

(g-7(C)g(C) = - ^^^'t'° + '°^ sC-^ 



This implies that 

M\QM-\0 



2 + ^' + ?/■ + W' 



^^C' + o{\c'\). 



Then, by Liouville's theorem, the expression (4.2) is identically equal to zero. Mul- 
tiplying it by C and passing to the limit C — > oo we see that 



2m . / 

A + B + Cj ^ 2 "^3 = 0. 

The remaining properties of A and B follow directly from their definitions. This 
concludes the proof of the Theorem modulo Lemma 4.7. 
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Proof of the Lemma 4 ■7- Let us give a proof for an odd value of j. The proof for 
the even j's is obtained by changing the sign of We will omit the subscript "j" 

in ttj and Cj. 

Near the point a the jump matrix for M has the form (Lemma 4.5) 

(M_w)-'AM.) ^ { 

where f° = C^f, g° = C^^g are locally constant vectors, and C° = C^C^^ is a 
locally constant matrix. Note that {{C°)-'^ ffg" = f^g = 0. 
Set 



M(C) = 

Then 

{M_{x))-^M+{x) = 
and we note that 



M(C), > 0, 

M(C)C°, $>C<0. 

/ + 27^^(C"')-V°(^°)^ a;<a. 

/, X > a, 



M°(C) 



M° = exp HI + 27ri (C")" V°(^7°)') ln(C - a; 

= eMiC^r'ng^y HC -«)) = /+ (C°)- ln(C - a) 

is also a solution of this local RHP. (Here we use the fact that {{C°)~^ f°{g°Y) = 
0.) Hence, 

' M°(C), > 0, 

M°(C)(C°)-i, QC<0, 

is a local solution of the RHP for M near ( = a. 

Since M = mjmC~^ ^ m and arc bounded near a, and mj is square inte- 
grable near a (Proposition 4.1), we conclude that M is square integrable near a. 
Clearly, M° is also locally square integrable, and detM° = 1. Hence, Ha{C) = 
M{(){M°{())~^ is locally integrable and does not have any jump across R near a. 
Therefore, Ha{C) is holomorphic near a. Since detM = 1, i^^ is nonsingular. We 
obtain 

M(C)-i^„(C)M°(C). (4.3) 
Computing M'M~^ explicitly we arrive at the desired claim with 

h = H',H-^ and C = Ha{a) {C<^)-^ f°g° H'^a). 

Since {C°)^^ f°g° is nilpotent and nonzero, the proof of Lemma 4.7 and Theorem 
4.6 is complete. □ 

Remark 4.8. Arguing in exactly the same way as we did in the proof of Theorem 
4.6 above, it is not hard to prove the equation 

with some constant matrices Aq, Bq. These matrices can be explicitly computed 
(as opposed to the matrices A, B, {Cj} in Theorem 4.6!). The resulting system of 
differential equations is equivalent to (2.5), (2.6), (2.7). 
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5. General setting 

As noted earlier, the arguments which we used to derive Theorem 4.6 and which 
we will use to derive further results, can be applied to other kernels as well (see §8 
for examples). In this short section we place the results of the previous section in 
a general framework. 

Let K{x,y) be a smooth integrable kernel 

^ y i=i 

defined on a subset X of the real line. Let us assume that X is finite union of 
(possibly infinite) disjoint intervals. 

We list the conditions on the kernel K that we need. 

(1) Assume we are given functions {/j, gj}jLi on 3C for which there exists a solution 
m of the RHP (X, v) with the jump matrix 

v = I + 27ri fg* = [Sm + liri fkgi]k,i=i 

such that the relations F = m+f — rri-f, G — m'^g — mZ^g are satisfied. 
Then necessarily 

N 

J2M^)gji^) = fix)g{x) = {mf)\x){m-'g){x) = F\x)G{x) = 0. 

For such functions {fj,gj}f^i the kernel L{x,y) defined by 

. J2f=ifj{x)gj{y) 

L[x, y) = , 

x-y 

formally satisfies the relation K = L(l + L)~^, see Proposition A. 2. As we have 
seen in §3, it can happen that the solution m of the RHP is defined but the integral 
operator L in L'^{3C,dx) given by the kernel L{x,y) is unbounded. In such cases, 
greater care must be taken in assigning a meaning to L(l + L)~^. 

Let J be a subset of X formed by a union of finitely many possibly infinite disjoint 
intervals: 

J = (ai, 02) U • • • U (a2m-i, a2m) C X, 
—00 < ai < 02 < • • • < a2m < +00. 

The endpoints {aj} of J are allowed to coincide with the endpoints of X. 

(2) Assume that the kernel K"^ = K\j defines a trace class integral operator in 
L'^{J,dx). 

(3) Assume also that the operator 1 — K"^ is invertible in L^(J, dx). 
Further, 

(4) Assume that the restrictions of the functions Fj,Gj to J lie in LP{J,dx) fl 
L°°{J, dx) for some p, 1 < p < 00. 
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Then by Proposition A. 2 there exists a solution mj of the normahzed RHP 
(J, vj) with vj = I- 2TTiFG\ and the kernel of the operator R-^ = K-^il - K^)-^ 
has the form 



R\x,y) 



x-y 

^ = {mj)±F = {mj)±m±f, Q = {mj*)±G = {mf)±ml^g. 

Set mx\j — mjm. As in §4, we see that mx\j satisfies the RHP {3C \ J,v). 
The crucial condition is that this RHP can be reduced to a RHP with a piecewise 
constant jump matrix. We formulate this more precisely as follows. 

(5) Assume that there exists a matrix valued holomorphic function C : C \ 6 — > 
Mat(iV,C) such that 

(a) C is invertible; 

(b) f° = C-f is a piecewise constant vector on X; 

(c) g° = C^*g is a piecewise constant vector on X; 

(d) C° = C-C^^ is an invertible piecewise constant matrix on X; 

(e) (C~^)'(C)C(C) = D(^~^ + o(|C|~^) as C ^ oo, where D is a constant matrix. 

Now form the matrix M = mx\jC~^ = mjmC~^ . Condition (5) implies that 
the jump matrix for M, which is equal to 

V = MI'M+ = C° + 27rir{gyxx\J 

(cf. Lemma 4.5) is piecewise constant. 

Now in order to ensure the existence of a differential equation for M with respect 
to C we need to know something about the local behavior of M near the points of 
discontinuity of V and near infinity. 

To state the condition on the local behavior of M we have to be sure that the 
matrix exists. Note that the determinants of v and vj are identically equal 

to one, because both v and vj are equal to the identity plus a nilpotent matrix. 
This means that the scalar functions det m and det mj have no jump across X. As 
m and mj tend to / at infinity, detm and det mj tend to 1 at infinity. Modulo 
certain regularity conditions on detm and det mj near the endpoints of X and J 
(which are always satisfied in the applications), Liouville's theorem implies that 
detm = detm J = 1, and the matrices m, mj, and M are invertible. 

(6) Assume that M'{C)M-^{C) = 0(|C - a]"^) at any endpoint a of X. 

(7) Assume that TO3e\j(C)™x\j(0 — '^(ICI"''") as ^ oo (recall that mx\j = mjm). 

Before going any further, we indicate where we proved that the conditions (l)-(7) 
hold for the 2F1 kernel. 

The condition (1) is verified in Proposition 3.2; (2) follows from Propositions 
2.7 and 2.8; (3) is Corollary 3.7 (here we needed additional restrictions on the 
parameters z, z' ,w,w'); (4) is a corollary of (2.8), (2.9); (5) consists of obvious 
properties of the matrix (3.3); (6) follows from Proposition 3.3 and 3.4; (7) is a 
corollary of (2.8), (2.9), and Proposition 4.2. 

Denote by {bj}^^i C C all finite endpoints of X and J. 
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Theorem 5.1. Under the conditions (l)-(7) above, there exist constant matrices 
{Bj}'^^^ such that the matrix M satisfies the following linear differential equation: 

n „ 

^'(0 = E73T- -^(O- (5-1) 

If bj is an endpoint of J but not an endpoint of X then the corresponding matrix 
Bj is nilpotent and nonzero. Moreover, Yl]j=i — ^> where the constant matrix 
D is given in (5e). 

Proof. We will follow the proof of Theorem 4.6. Since M has a constant jump 
matrix, the matrix M'M~^ has no jump across X. If bj is an endpoint of X then by 
(6), bj is either a regular point or a first order pole of M'M~^. If bj is an endpoint 
of J and not an endpoint of X then the proof of Lemma 4.7 (which can be repeated 
word-for-word in the general setting) shows that near bj 

B- 

M' {QM~^{Q — - — — h a locally holomorphic function 

C ~ bj 

with a nilpotent constant matrix Bj. Thus, 

B 

M'(C)M-\C) - E TZTT (5.2) 



3 



is an entire function for the (constant) matrices {Bj}^^^. 
Near ( — oo, 

M'M-' = m'x^jm-lj + mx\j{C-'yCm^{j = DC'+ o(|C|-'), 

as follows from (5e) and (7). Hence, by Liouville's theorem, the function (5.2) 
is identically zero, and computing the terms of order at infinity we see that 
Y.UBj = D. □ 

Remark 5.2. Arguing as above and replacing the condition (7) by the estimate 
m'm~^ = o{\C,\~^) s& C, ^ oo., one can easily prove that 

{m{OC-\C))' = Y,^ •m(C)C-^(C), 

where [b^Yj^i are the endpoints of X, and {B'j} are some constant matrices, cf. 
Remark 4.8. 

If we allow the differential equation to have an irregular singularity at infinity, 
then the condition (5e) on the matrix C can be relaxed. Let us introduce the 
condition 

(5e') {C~^)'{C,)C{C) = -D + o(l) as C — oo, where D is a constant matrix. 

We can then relax the condition (7) to 
(7') Assume that t^'x\j^O'^'x\j^'0 ^ ^^^) as Q ^ oo. 

The following claim is proved in exactly the same way as Theorem 5.1. 

35 



Theorem 5.3. Under the conditions (l)-(4), (5a-d), (5e'), (6), (7') above, there 

exist constant matric 
differential equation: 



exist constant matrices {Bj}j'^i such that the matrix M satisfies the following linear 



^'(0= IE73T- + ^|^(C)- (5-3) 



If bj is an endpoint of J but not an endpoint of X then the corresponding matrix 
Bj is nilpotent and nonzero. 

Remark 5.4. Once again, if m'm~^ ^ as C ^ oo, then 

(m(C)C-i(C))' = [y.^ ^ + m(C)C-i(C), 

where [b^Y-^^ are the endpoint of 3£, and {-B^} are some constant matrices, cf. 
Remarks 4.8, 5.2. 

6. ISOMONODROMY DEFORMATIONS. JIMBO-MIWA-UENO T-FUNCTION 

Let M(C) be a matrix- valued function on the complex ^-plane satisfying a linear 
differential equation of the form M'{C) = S(C)M(C), where B{C) is a rational 

matrix. 

Fix a fundamental solution M of this equation. In general, M{C) is a multivalued 
function. If . . . , bn} are the poles of -B, then . . . ,bn, oo} are the branch 
points for M. When we continue M along a closed path 7 avoiding the branch 
points, the column vectors of M are changed into some linear combinations of the 
columns of the original matrix: M(C) 1— > M{QX^. Here Xy is a constant invertible 
matrix depending on the homotopy class [7] of the path 7. Thus, Xj^s provide a 
"monodromy representation" of the fundamental group of C \ {61, . . . , bn}: 

X : 7ri(C \ {61, . . . , bn}) ^ GL{N, C), [7] ^ X^. 

Now view the singular points {61,..., 6^} as variables. It may happen that 
moving these points a little and changing the rational matrix -B(C) in an appropriate 
way, we do not change the monodromy representation. In such a case we say that 
we have an isomonodrom,y deformMion of the initial differential equation. 

For general information on isomonodromy deformations we refer the reader to 
[IN], [JMU]. 

Without loss of generality, we can assume that, in the notation of §5, the first 
k < n points {61, ... , bk} of the set {bj}j'^^ are exactly those endpoints of J which 
are not the endpoints of X. Clearly, {bj}^^^^ C {aj}'^'^^. 

The following statement is immediate. 

Proposition 6.1. Under the assumptions of Theorem 5.1 (or Theorem 5.3), there 
exists e > with the property that moving the points bi, . . . ,bk within their e- 
neighborhoods inside M provides an isomonodromy deformation of the equation (5.1) 
(or of the equation (5.3), respectively). 

Note that the matrices {Bj}^^i are now functions of 61, . . . , 6^- 
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Proof. Choose e > so that the points 61 , . . . , 6/. cannot coUide between themselves 
or with the other endpoints bk+i, ■ ■ ■ ,bn- Since the matrix M = mjmC~^ has 
nonzero determinant, this matrix can be viewed as a fundamental solution of (5.1). 
The monodromy of this solution, as we go along any closed curve which avoids 
the singular points, is equal to the product of the values of the jump matrix V or 
their inverses at the points where the curve meets X. Since V does not depend on 
bi, . . . ,bk, the proof is complete. □ 

In 1912, Schlesinger realized that if the matrix B{() has simple poles then a 
deformation of 6j's preserves monodromy if and only if the residues {Bj} of B at 
the singular points, as functions of 6j's, satisfy a certain system of nonlinear partial 
differential equations. These equations are called the Schlesinger equations. The 
analogs of the Schlesinger equations in the case when B has higher order poles were 
derived in [JMU]. 

In what follows we will use the Schlesinger equations arising from the isomon- 
odromy deformation described in Proposition 6.1. Since our situation is simpler 
than the general case in [JMU] , it is more instructive to rederive the equations that 
we need, rather than to refer to the general theory. 

Proposition 6.2 (Schlesinger equations), (i) The matrices {Bjj'j^i from (5.1), 
as functions in bi, . . . ,bk, satisfy the equations 

dBi _ [Bj,Bi] dBj _ ^ [Bj.Bi 



dbj bj — bi dbj ^ bi — b. 

KKn 



<«<i 
1^3 



where j = 1, . . . ,k, I = 1, . . . ,n. 

(a) The matrices from (5.3), as functions in bi, . . . ,bk, satisfy the equa- 

tions 

dBi _ [B,M dB, _ ^ [B,,Bi] _ 
dbj~b,-br dbj~ ^ bi-bj 

l<l<n 

where j = 1, . . . ,k, I = 1, . . . ,n. 

Sketch of the proof. Since M satisfies a RHP with a constant jump matrix V, the 
derivative M^. = satisfies the same jump condition, j = 1,. . .,k. Hence, the 

matrix M^. has no jump across X. Thus, it is holomorphic in C \ {bj}. As 
was shown in the proof of Lemma 4.7, locally near ( = bj we have 

M(c) = HiOeMicTTig^Y HC - bj)) 

= H{0 {l+{C")-'fo^goY ln(C-6,)), 

where H is holomorphic. With some additional effort, one can show that H is 
differentiable with respect to bj, and differentiating with respect to bj we see that 

M,(C)M-(C) = ^ +0(1). 

Since M ~ / at C = 00, one can show that Mf).M~^ — > as — > 00. By Liouville's 
theorem, M^.M'^ + Bj/{z - bj) = 0, and 

Mb, = --^M. (6.3) 
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The linear equations (5.1) and (6.3) form a Lax pair for (6.1). 

Differentiating (5.1) with respect to bj and (6.3) with respect to (, subtracting 
the results, and multiplying the difference by on the right, we obtain 

2^ Ob. c-bi c-bj ^ C-bi • 

'=1 l<l<n 

The equality of residues at the points on both sides of this identity gives 

(6.1). The equations (6.2) are proved in exactly the same way. □ 

Corollary 6.3. In the notation of Theorem 4-6, 



da, friaj-y da, a, + | ' ^ 

dCi _ [Cj,Ci] dCj _ [Cj,A] [C„B] ^ [C„Ci] 
daj aj — ai oaj aj — i + f o-j — cti 

l<l<2m 

Here j,l = 1,2, ... ,2m, and if ai = — oo or a2m = +oo then the corresponding 
terms and equations are removed. 

Proof. Direct application of Proposition 6.2. □ 

It is known that for any solution of Schlesinger equations there exists an asso- 
ciated remarkable 1-form cj which is closed, see [SMJ], [JMU]. For the equations 
(6.1), the form of a; is as follows: 

i=l l<Z<n 

while for the equations (6.2) the form is different: 

/ \ 

Tr{BjB, 



k 



. l<Z<n , 



dbj. (6.7) 



Definition 6.4 [JMU]. A function r = t(6i, . . . , bk) is called a r-function for the 
system of Schlesinger equations (6.1) (or (6.2)j if 

d\n.r = u) 

with uj given by (6.6) (or (6.7), respectively). 

The definition can be extended to the most general case of an arbitrary rational 
matrix B{Q, see [JMU]. 

Since da; = 0, the r-function is defined at least locally. Clearly, the r-function is 
unique up to a multiplicative constant. 

The following claim is a corollary of much more general statements proved in 
[Miw] and [Mai]. 
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Painleve property. Any solutions {Bj}'j^^ of the Schlesinger equations (6.1) or 
(6.2) are analytic function in {bi, . . . , bk) which have at most poles in addition to 

the fixed singularities bj = bi for some j ^ I. 

The corresponding T-function is holomorphic everywhere on the universal cover- 
ing manifold of 

& \ {(61, ...,bk)\bj = bi for some j ^ I, j = 1, . . . ,k, l = l,...,n}. 



Let us now return to the general setting of §5. The next statement is our main 
result in this section. 

Theorem 6.5. Under the assum,ptions of Theorem 5.1 {or Theorem 5.3), the Fred- 
holm determinant det(l — K'^) is the T-function for the system of Schlesinger equa- 
tions (6.1) (or (6.2), respectively) . 

Proof. We will give a proof under the assumptions of Theorem 5.1, the case of 
Theorem 5.3 is handled similarly. 

First of all, by condition (2) of §5 the operator is trace class. Hence, det(l — 
K^) is well-defined. Note that (1 — K^) is invertible by condition (3). By a well- 
known formula from functional analysis, we have that 

aindet(l -K-^) , ^ 

'- = ±R\b,,b,), j = l,...,k, 

where R"^ — {1 — K'^)~^, the sign "+" is chosen if bj is a left endpoint of J, 
and the sign "— " is chosen is a right endpoint of J. Thus, in order to verify that 
(ilndet(l — K"^) = oj we must prove that 

R'(b,,bj)= y: ^^^^ j=i,---,k. (6.8) 

l<l<n ^ 

We give a proof when bj is a left endpoint of an interval from J. The proof for 
the right endpoints is obtained by changing the sign of Q. 
We have 

x,y^bj X — y x,y^bj X — y 

{{M+C+)-'g{y)YM^C^f{x) _ ^.^ {g-fM-\y)M_{x)f'^ 

x,y^bj X — y x,y^bj X — y 

= {g")\M-'M'_){b,)f". 

The local representation (4.3) of the matrix M(C) near the point C, = bj implies 
that 



M(C) 



H,^ (C) exp ((C°)- V°(^7°)* ln(C - h,)) , > 0, 

ifb,(C)exp ((C")-ir(^°)Mn(C - b,)) {C")-\ »C < 0. 
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Hence, foix&X near bj , 

= exp {-{C^-'ngn' Hx - b,)) H-\ 
M'_ = H',^ exp ((C°)-V°(^°)Mn(x - bj)) (0°)"^ 

X - 6j 

+ exp (-(C°)-V°(^°)*ln(a; - 6,)) i^.-^i^^. exp ((C^)" V°(^'')Mn(x - b^)) {C")-^ 

where Hi,, is a function holomorphic near bj, and dei Hi).{bj) ^ 0. 
Since (^°)*(C°)"V° = ((C°)~V°)V = 0, we have 

X — bj 

(^7°)*exp(-(C'')-V°(^7°)Mn(x-6,)) = {g")\ 
exp ((C"')-V°(^°)* ln(x - bj)) {C°)-^r = {C")-'r- 

Therefore, 

R{bj,bj) = {g")\M-'M'_){bj)r = {9")\H^'Hl^){bj){CT'r- (6-9) 

On the other hand, let us compute the right-hand side of (6.8) through C°, /°, 
g°, and Hf, .. As above, locally near bj we have 



M\C)M-'{C) = H^C)H^'{0 + 



Hb,{0{C")-'ng")'H-MO 



C-bj 

Comparing with (5.1), we conclude that 

Bj = H,^ib,){C'^)-'ng")'H-^ib,) 

and 

l<Z<n 

-H,^ib,)iCn-'ng")'H-^ib,)H^b,)H-^ib,). 

Multiplying these two relations, taking the trace of both sides, and using the 
fact that {g°)\C")-^f" = 0, we obtain 

Tr{H',^{b,){C'^)-'ng")'H;^{b,)) ^ E 



bi - bi 

1 

1^3 



'j - 01 

KKn 



But the left-hand side of the last equality equals the right-hand side of (6.9). This 
concludes the proof of (6.8). □ 
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Corollary 6.6. Let K he the continuous 2F1 kernel of ^2 and assume that 



z + z' + w + w'>0, \z + z'\<l, \w + w'\<l. 

Then, in the notation of Theorem 4-6, det(l — i^"^) is the r-function of the Schlesin- 
ger equations (6.4), (6.5), where the matrices A, B, {Cj}'j^i satisfy the conditions 
stated in Theorem 4-6- 

Proof. Direct application of Theorem 6.5. □ 

Note that the restrictions on the parameters z, z' , w, w' come from CoroUary 3.7 
(see also Remark 3.8). 

7. Painleve VI 

In this section we consider the case of the 2-^1 kernel acting on J = (s, +00) for 
s > |. We will show that the Predholm determinant det(l— Kg) = det(l— _|_oo)) 
can be expressed through a solution of the Painleve VI equation. The appearance 
of the P VI equation is to be expected from the general results of [JMU] ; the precise 
form of the equation is not clear in general, and requires considerable calculations, 
as we now show. 

Our goal is to prove the following claim. 

Theorem 7.1. Let Kg be the restriction of the continuous 2F1 kernel to the interval 
(s, +00), s > |. Assume that & = z -\- z' -\- w -\- w' > \z + z'\ < 1, \w -\- w'\ < 1. 
Then the function 

/ iw IX dlndet(l - Ks) 2 , i^3Z^4 

^(^)=(^-2) (^+2) ""1^+^ 

satisfies the differential equation 

v2 



+ \) ^") = (2 {sa' -a) a'- v^v^v^v^y 



(7.1) 



where 



z -\- z' -\- w -\- w' z — z' + w — w' z — z' — w + w' 

Ul = V2 = 7: , 1^3 = , 1^4 = 



Remarks 7.2. 1. The equation (7.1) is the so-called Jimbo-Miwa cr-version of the 
Painleve VI equation, see [JM, Appendix C]. It is easily reduced to the standard 
form of the Painleve VI, see [JM], [Mah]. 



2. As s — > +00, 

dlndet(l-Ks) 
ds 



~ K{s,s) = iPout{s){K,,{s)Sout{s) - S',,,{s)Rout{s)). (7.2) 



The error term in this asymptotic relation is of order °° K{s, y)K(y, s)dy. Using 
the leading asymptotic terms 



sin TTZ sin ttz' 



V'out(s) ~ Z3. ^ ^out(s) ~ 1, S'out(s) ~ const s \ 
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7r2 



we see that K{s, s) = 0(s-®-2), and /+°° K{s, y)K{y, s)dy = 0(s-2®-3). Hence, 



a(s) = -ufs + 



+ 



Sm TTZ Sm TTZ 



-2i/t 



+ 0(s-2-i). 



2 TT^ 

This expansion determines a{s) uniquely as a solution of (7.1) by a result of 
O. Costin and R. D. Costin [Cos]. 

3. The restrictions 6 > 0, |2;+2;'| < 1, and Iw+tu'l < 1 are taken from Corollary 3.7. 
Most likely, they can be removed from Corollary 3.7, and hence from Theorem 7.1, 
see Remark 3.8. Another possible way of removing these restrictions from Theorem 
7.1 is to prove that the Fredholm determinant det(l — Kg) and its derivatives with 
respect to s, which are well-defined for all admissible sets of parameters (see the end 
of §2) are real-analytic function of the parameters. Then the result would follow 
by analytic continuation. 

4. The equation (7.1) depends only on 3 independent parameters: the shifts 



z ^ z + a, z' ^ z' + a, 



w ^ w — a. 



w 



w 



a 



do not change the values of z^i, . . . , zy4. However, the solution of (7.1) which is of 
interest here, depends nontrivially on all four parameters, as can be seen from the 
above asymptotic expansion. 

5. The proof of Theorem 7.1 follows the derivation of the Painleve VI equation 
from Schlesinger equations given in [JM, Appendix C], see also [Mah] for a more 
detailed description. 

Proof of Theorem 7.1. By Theorem 4.6, the matrix M satisfies a differential equa- 
tion 



d 



M(C) 



A 



+ 



B 



+ 



M(C) 



with some constant matrices A, B, and C, such that 

Tr^ = TrB = TrC = 0, 



detA = - 



z — z 



detB = - 



w — w 



detC = 0, 



and 



6 

A + B + C = -y^s, 



1 
-1 



By Corollary 6.3, the matrices A, B, C satisfy the Schlesinger equations 



dA 
ds 



[C,A] 



dB 



s — 



s + 



(7.3) 
(7.4) 



Introduce the notation 



w — w 



Set 
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Lemma 7.3. a{s) = Tr [{{s + I) A + {s - ^) B) C] . 
Proof. Follows from (6.6), Definition 6.4, and Corollary 6.6. □ 
Write the matrices A and C in the form 



A = 




XA 






Xc 




VA 


-ZA 




yc 


-zc 



with 

XAVA = -detA- z^^ Oj^- z% 

Lemma 7.4. a' = —&zc- 
Proof. Lemma 7.3 implies 



xcyc — — detC — z"^ — —z"^. 



(7.5) 



(7.6) 



a'{s) = Tr((^ + B)C) + (s + Ty{A!C) 

+ (. - i) T.{B'C) + [((. + i) ^ + (. - i) i3) C] . 

The Schlcsingcr equations (7.3), (7.4) imply that last three terms vanish due to the 
identity Tr([X, Y]X) = 0. Further, since A + B = -f (73 - C and = 0, we have 



a'{s) = Tr 



e 



as-C]C] = 



6 



Tr((73C) = -ezc. □ 



Lemma 7.5. 

- I) + I) ^"(^) 



6 

— Tr(cT3[^,C]) = e{xcyA- XAVc)- 



Proof. Differentiating the equality &'{s) = — ® Tr(a"3C) and using the equation 



(7.4) we get 



e 



(. + i) a"{s) = --Tr{a3[{s+'^)A+{s- i) C] ) . 

Substituting B = — ®cr3 — ^ — C and simplifying we arrive at the first equality. The 
second equality follows from the explicit form of matrices A and C, see (7.5). □ 

Lemma 7.6. (s - |)(t'(s) - a{s) = -Tr(^C) = -{xaVc + xcVa + '^zazc)- 
Proof. We have 

(. - i) a'{s) - a{s) = -!(.- I) TriasC) - Tr [{{s + 1) ^ + (. - i) B) C] 



6 



= -Tr {AC), 



& 

where we used Lemmas 7.3, 7.4 and the relations B = ——a3 — A — C and = 0. 
The second equality follows from (7.5). □ 
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Lemma 7.7. (s + ^)a'{s) - a{s) = Bza + 0^-9% + &'^/A. 
Proof. We have 

- Tr(^C) = Tr (73 + B + ^ 



= &ZA + &ZC + e\-el + 6V4, 

where we used the equahties 

TiA^ = 2e% TiE^ = 2el, C^ = 0, Tral = 2. 
Lemmas 7.4 and 7.6 conclude the proof. □ 

Now we use the foUowing trick to derive the differential equation for a. We 
learned this trick from [JM] in which the authors refer further to [Oka]. 
From Lemmas 7.5 and 7.6 we know that 

xcVA - XAVc = - I) (s + I) ^"{s), 

-{xaVc + xcVa) = - ^'(s) - a{s) + 2zAZc- 

Squaring these equalities and then subtracting the first one from the second one, 
we obtain 

'^XAXcVAVc = {{s - I) ^'{s) - &{s) + 2zAZcf 
But (7.6) implies that xaXcVaVc = {z'a ~ ^a)^c- This gives 

4(4 - 0a)4 = ((« - 1) ^'i^) - H^) + '^^Azcf 

Next, Lemmas 7.4 and 7.7 provide expressions for zc and za via (t(s). Namely, 
a'{s) 



Zc = 



6 



ZA^^(^{s + ^)a'{s)-a{s)-e\ + el-^y 



Substituting these relations into (7.7) we can obtain a differential equation for a. 
But we can also rewrite everything in terms of cr(s). We have 



a"{s) = a"{s), 
{s - i) a'{s) - a{s) = (. - i) a\s) a{s) + ^-^ " ^« 



e\ -el 6^ 

8 ' 



6 
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Substituting this into (7.7) we have 



{s-\)a'{s)-a{s)+^\^-- 



& 



[s + \)a'{s)-a{s)- 



a'{s) + 



Purely algebraic manipulations show that the equation above after the multiplica- 
tion by &^a'{s) turns into the equation (7.1). Note that in this notation 
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8. Other kernels 

8.1. The Jacobi kernel. We introduce some notation related to the Jacobi 
polynomials. Our notation follows [Er, 10.8]. 

Let {Pn — Pi"''^\x)}'^^Q be the system of orthogonal polynomials on (—1, 1), 
degP^ = n, with respect to the weight function w{x) = (1 — a:)"(l + x)^, where 
a and (3 are real constants, a, P > —1. The normalization is determined from the 
relation 

+ a\ {a + 1)^ 



^'n(l) . , , 

\ n J n\ 

where (a)fc = T{a + k)/T{a) is the Pochhammer symbol. The PnS are the Jacobi 
polynomials with parameters a and f3. Let us denote by the square of the norm 
of Pn in L^((— 1, l),w{x)dx) and hj kn > the highest coefficient of 

hn = J Pn{x)w{x)dx, Pn(a;) = A;n a;"' + { lower order terms }. 

The explicit form of these constant is known, see [Er, 10.8], 



2«+/3+ir(^ + a + l)r(?i + /3 + 1) 



kn — 2 



_^f2n + a + /3 



n 



(2n + a + /3 + l)n!r(n + a + /3 + 1) 

The Jacobi polynomials are expressed through the Gauss hypergeometric function 
2F1 



Pn{x) 



n + a 



n 



-n, n + q: + /9 + 1 

Q!+ 1 



1 — X 



The Jacobi functions of the second kind Qn{x) = Qn''^\x) are defined by the 
formula 

2"+"+'^r(n + a + l)r(n + (3 + I) 



Qn{x) = 



r(2n + a + /3 + 2) 
X (a;-l)-"-«-^(a; + l)-^2Fi 



n + l, n + ct + l 
2n + CK + /3 + 2 



1 — X 
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Qn satisfies the same second order differential equation as P^. The Jacobi functions 
of the second kind are related to the Jacobi polynomials by a number of well-known 
formulas, see [Er, 6.8], [Sz, §4.6] for details. 

Proposition 8.1. For any n = 1,2, . . ., take two arbitrary integers k and I such 
that k + I = n, and set 

z = k, z' = k + a, w = I, w' = I + p. 

Then 

. . _ {'i--'2xf''^'^{'i- + '2x)^^+'^ _ {l-2xf^{l + 2xf^w{2x) 

Proof. Follows from the direct comparison of formulas. The relation 

sm{7Tz')T{z-z') {-!)'' sm{-K{z' - z))T{z - z') (-1)'= (-1)'= 



TT TT T{l + z'-z) r(l + a) 

should be used along the way. □ 

Remark 8.2. The functions -Rout and 5'out can be similarly expressed through 
Qn-i and Qn, respectively. Since, we do not use the corresponding formulas below, 
we leave their derivation to the interested reader. 

The nth Christoffel-Darboux kernel for the Jacobi polynomials is given by the 
formula 

r^CDf ^ Pjix)Pjiy) kn-1 Pn{x)Pn-liy) - Pn-l{x)Pn{y) 

(a;,y) = 2^ = TTT • 

We define the nth Jacobi kernel on the interval ( — |, |) by the formula 



Ki-%x,y) = 2K^''{2x,2yWw{2x)w{2y), x,y e (-i, |) . 

The corresponding integral operator K^^^ is the orthogonal projection in L^((— |, ^), dx) 
onto the n-dimensional subspace spanned by 

^(i_^)t a:-' - x)^'^ + x)^ . 

Proposition 8.3. Under the assumptions of Proposition 8.1, 

-K^outjOut ~ -^outjin ~ -^in,out ~ 0; -^in,in — i 

where K is the continuous 2F1 kernel. 

Proof. The vanishing follows from the vanishing of ipout, which, in turn, follows 
from the vanishing of sin 772; and sinTrtu. The equality i^in,in = ^i^^ follows from 
the definition of both kernels and Proposition 8.1. □ 

Thus, the Jacobi kernel can be viewed as a special case of the 2F1 kernel. Our 
next step is to extend the results of §6 and §7 to this kernel. 

Let J — (ai, 02) U ■ ■ ■ U (a2m-i, ^2™,) be a finite union of disjoint intervals inside 
(— |, ^). It may happen that ai = — | or a2m = |- However, we require J to be a 
proper subset of (— ^, |). 
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Proposition 8.4. Assume that 0<Q!<|, 0</?<|. Then the Jacobi kernel 
satisfies the conditions (l)-(7) o/§5. 

Proof. (1) follows from the fact that K^^^ coincides with the 2F1 kernel for a specific 
set of parameters (Proposition 8.3), and for that kernel the condition was verified 
in Proposition 3.2. 

(2) is obvious, since K^^"^ is a finite rank operator. 

(3) follows from the fact that K^^'^ is a projection on a finite-dimensional space, 
and the range of this projection intersects L^{J,dx) trivially (here we used the 
condition that J is a proper subset of (— ^, ^)). 

(4) follows from the explicit form of the kernel (here we use the condition a, (3 > 
0, which guarantees the boundedness near the points ±|). 

(5) is exactly the same as for the 2F1 kernel. 

(6) is the only nontrivial condition. If ai 7^ — | and a2m 7^ | then the claim 
follows from Propositions 3.3 and 3.4, as for the 2F1 kernel. Now assume that 

— Since < /3 < -i. Proposition 3.4 implies that m{()C~^{C) is locally 



ai 



in L'* on any smooth curve passing through ( — By Proposition A. 2, mj i 



IS 



locally L^, hence, M — mjmC ^ is locally in L^/^. 

The jump matrix V for M locally near — | coincides with the jump matrix 
C° = C_C^'^ = C^'^C_ for C-i, see §5. This means that ii'(C)C"^(C) is a local 
solution of the RHP for M for any locally holomorphic H{C). Set M° = HC'^ 
with 



^(0 



(C+l)-" 



Note that H has no branch at — |, because w = I e Then 



M°(C) = 



(C-i)-'^(C + |) 







l)^(C+|)^ 



Hence, {M° (as well as M°{Q) is locally in L"*, because w' 
Thus, M{M")~^ is a locally function with no jump across M. 
near C = | 



-w = (3e (0,i). 
This means that 



M(c) = iy_i(C) 



(C + i)^ 

for some locally holomorphic function H_i{() such that H_i{ — ^) is nonsingular. 
Hence 

B 

M'M~^{() = Y + ^ locally holomorphic function , 

where B has eigenvalues (3/2 and —/3/2. 

The argument in the case a2m = | is similar, and the eigenvalues of the residue 
A of M'M~^ at C I are equal to a/2 and -a/2. 

Finally, the condition (7) for the Jacobi kernel follows from that for the 2 -Pi 
kernel. □ 

Now, by Theorem 5.1, for a,(3 E (0,^), the matrix M corresponding to the 
Jacobi kernel, satisfies the differential equation (cf. Theorem 4.6) 



M'iO = 



A 



+ 



B 



2m 



a,- 



M(C) 
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for some constant matrices A, B, and {Cj}'j'^i. If ai = — | then Ci = 0, and if 

a2m = \ then C2m = 0. 
Moreover, 

Tr^ = Tri3 = TrCj =0, 
det A = -aV4, det B = det Cj = 0, 

for all J = 1, ... , 2m, and 

2m 



^ + B+2^Cj = -n -^(73, (73 = 



1 
-1 



Further, by Proposition 6.2 the matrices A, B, {Cj}j'^i, satisfy the Schlesinger 
equations (6.4) and (6.5). Finally, Theorem 6.5 implies 

Theorem 8.5. Assume that < a, /? < |. Then the Predholm determinant 
det(l — K^^^\j), where K^^'^ is the Jacobi kernel, is the r -function for the sys- 
tem of Schlesinger equations (6.4), (6.5) with matrices A, B, {Cj}^^i satisfying the 
conditions stated above. 

Similarly to the 2F1 kernel, the cases when J = (— i, s) or J = (s, |) lead to the 
Painleve VI equation. Note that there are no restrictions on a and /3. 

Theorem 8.6 [HS]. Let Kg be the restriction of the Jacobi kernel K^'^ kernel to 
either the interval (— i,s), s < 1, or to the interval {s,\), s > —1. Then the 
function 

I 1. rflndet(l-Ks) / a + o? - ^'^ 

a(.) = (.-i)(. + i) ^-(- + ^J ^ + 

satisfies the differential equation 

where 

a + (3 a + (3 a — (3 

Proof. Simply repeat the proof of Theorem 7.1. Note that in this way we only 
prove the theorem for a, P e (0, |). But for the finite-dimensional Jacobi kernel 
it is obvious that the determinant det(/ — Kg) and all its derivatives depend on 
the parameters a and (3 analytically. That is the reason why we can remove the 
additional restrictions on a and /3. □ 

8.2. The Whittaker kernel. The Whittaker kernel, which we are about to 
introduce, plays the same role in harmonic analysis on the infinite symmetric group 
as the 2F1 kernel plays in the harmonic analysis on the infinite-dimensional unitary 
group, see §1. The problem for the infinite symmetric group was investigated by 
G. Olshanski and one of the authors in a series of papers, see [P.I-P.V], [BOl- 
3], [Borl]. For a brief summary we refer the reader to [BOl], [BOS, §3], [Borl, 
Introduction] . 
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Split the space jC = IR \ {0} into two parts 



Let z, z' be two complex nonintegral numbers such that either z' — z, or z and z' 
are both real and k < z, z' < k + 1 for some /c G Z. 
The functions 

are defined by the formulas 

ij+{x) = C{z, z')x-'-'' e-"" , ^P-{x) = (-x)"+"'e", 

where C{z,z') = smTTZsmirz' /tt'^, as before. The Whittaker kernel is a kernel on 
X, which in block form 

corresponding to the splitting 3£ = 36+ U 3£_ , is given by: 

x-y 
X y 

r, — -r^: — -r^ — S -{x)S Ay\ 

x-y 



x-y 



where 



z + z' — 1 a; 

= a; 2 e2 VF -^-z'+i z-z' (x) , 
2 ' 2 

5+(a;) = r(z + l)r(z' + 1) x^^et W^ _z-z'-i z^ (a;) , 

2 ' 2 

2 ' 2 

\.\Z)V\Z ) 1^1 

Here VFK,At(x) is the Whittaker function, see [Er, 6.9]. 

In the definition of the Whittaker kernel above we have switched the signs of the 
parameters z and z' ^ compared to the standard notation. The reason for the switch 
is the following. 

Proposition 8.7. The Whittaker kernel can he realized as a scaling limit of 
the 2F1 kernel , 

K^{x, y) = lim £ • (^+ex,^+ey) , x,yeR\ {0}, 

e— >+0 

49 



where for the 2F1 kernel we set w = s ^, w' = 0, and the parameters (z, z') for 
both kernels are the same. 

Proof. Using the well-known limit relations (a, 6, c e C) 

lim (1 + aef/' = e^ lim ga-b ^la + e ^) ^ 



lim 2F1 



a, h 


1 


l/£ + c 


ex 



a;"^2 e'^W -a-b+i a-b jx), a; ^ R_, 
2 2 



we see that (remember w = s ^, w' = 0) 

lim £^+^ Vout (I + ex) = X > 0, 

e— >+0 

lim^£~^~^ V'in (I +£a;) ='ip-{x), x < 0, 
lim^i?out (I + £x) = lim^£~^~^ S'out (| + = S+{x), x > 0. 

Further, if we identify i?+ and S+ with their analytic continuations, then on I 
we have 

1 — „ 1 R_i_ — 



= R- — ' + 
i/j- 27Ti ' '0- 27ri 

where for we denote by F+ and F~ the boundary values of a function F: 

F+{x) = F{x + iO), F-{x) = F{x -iO), 

see [Er, 6.5(7), 6.8(15), 6.9(4)]. Comparing these relations with (2.1), we conclude 
that, for a; < 0, 

lim i?in (k+£x) = R-(x), lim £'^+'^'5'i„ (k+sx) = S-(x). 

The result now follows from the explicit form of the kernels. □ 

Let J = (01,02) U ■•■ U (o2m-i)0i2m) be a union of disjoint, possibly infinite, 
intervals such that the closure of J does not contain the origin. 

Proposition 8.8. The Whittaker kernel satisfies the conditions (l)-(4), (5a-d), 
(5e'), (6), (7) o/§5 with the matrices 



C(C) = 







(- — e 



D = 

2 



1 
-1 



2 



Proof. The proof of (1) is very similar to the case of the 2-^1 kernel. The kernel 
L{x,y) has the form 

A' 
-A* 

where ^ is a kernel on x R_ of the form 

•\/'0_i_(a;)'0_(y) C(2, 2')x~'^^e~^(— l/)^^e^ 



z+z' 



A{x,y) 



x-y 



x-y 
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The jump condition is verified using the formulas [Er, 6.5(7), 6.8(15), 6.9(4)]. 



(2) can be either verified in the same way as for the 2F1 kernel, or it can be deduced 
from the fact that the Whittaker kernel is the correlation kernel of a determinantal 
point process which has finitely many particles in J almost surely (see [So, Theorem 
4] for the general theorem about determinantal point processes, and [BOl], [Borl] 
for the needed property of the Whittaker kernel). 

(3) If 1^ + < 1 then the kernel L introduced above defines a skew, bounded 
operator in L'^{X,dx), and K — L(l + L)~^, see [P.V], [BOl]. Then, similarly to 
Corollary 3.7, we can prove that 1 — K"^ is invertible. 

However, for the restricted operator K"^, we can prove the invertibility of 1 — K"^ 
for all admissible values of {z, z'). The following argument is due to G. Olshanski. 
Write K"^ in the block form 



Ki , Ki 

5~r ) 



corresponding to the splitting J = ( J n M+) U ( J fl R_). Since is a correlation 
kernel, K^ j^ and _ are positive definite. Moreover, K^ j^{x^y) = —K^ _{y^x) 
by definition of the Whittaker kernel. Thus, it is enough to prove the invertibility 
of 1 — K'^ ^ and 1 — K'L _ (see proof of Corollary 3.7). 

We consider K'^ _|_; the proof for K'l_ _ is similar. By [So, Theorem 3], < 1 

and K{^^ = < 1. The only way can have norm 1 (remember that 

is of trace class, hence, it is compact) is that has an eigenfunction with 

eigenvalue 1 which is supported on J fl R+. By [P.V, Proposition 3.1], see also 
[BOl], = i^+,+ (a;,y) commutes with a Sturm-Liouville operator 

_ d ^ d (z + z' + x)'^ 



dx dx 4 



in the sense that 



K+,+{x,y)Dy = DxK+^+{x,y) 

for all x,y > 0. Suppose / G L^(M+) is an eigenfunction of with eigenvalue 

1 and supported in J n M_|_, i. e., 

/ K+^+{x,y)f{y)dy^ K+^+{x,y)f{y)dy = f{x), x > 0. 

Then using the decay and smoothness properties of K_|_^_|_(a;, y), which follow easily 
from the known properties of the Whittaker function, one sees that D^f also belongs 
to L^{R+) and 

K+,+ {x,y)Dyf{y)dy = D^f{x). 



j 

JR. 



Thus 

V = Span{D^/ : A; > 0} C Ker(l - K^^^) C L^{^+). 

But as K^ j^ is compact, dimF < 00, and hence F is a finite dimensional invariant 
subspace for Dj^. It follows that there exists a nonzero v & V such that D^^v = Xv 
for some scalar A. But as v E V, it must vanish in a neighborhood of x = 0, which 
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is not possible for nontrivial solutions v{x) of the differential equation D^v = Xv. 
Thus, we obtain a contradiction, and hence .|.|| < 1 and 1 — is invertible. 
The proof of (3) is now complete. 

(4) and (5a-d), (5e') arc easily verified. The proofs of (6) and (7) are similar to the 
case of the 2F1 kernel, and we do not reproduce them here. □ 

By Theorem 5.3, the matrix M for the Whittaker kernel satisfies the differential 
equation 



^ 7=1 ^ 



■3 



The matrices {Cj}'^'^^ are nilpotent (if ai = —00 or = +00, then Ci = or 
C2m = 0, respectively), and an analog of Proposition 3.3 shows that 



/\ 2 

z — z 



Tr^=0, det^=- 

Theorem 8.9. The Fredholm determinant det{l — K\j), where K is the Whittaker 
kernel, is the r-function for the system of Schlesinger equations 



da^ ^ tti — h ^ daj ^ a,- — a; 2 

■' J=l ■> ^ J J l<l<2m 



The matrices {Cj}^'^i are nilpotent (if ai = —00 or a2m = +00, then Ci = or 
C2m = 0, respectively), and 



Tr^ = 0, det^ = 



2 

z — z 



Proof. These results follow from Theorem 6.5. □ 

The next step is to consider J = (s, +0x3), s > 0. It turns out that in this case 
the Schlesinger equations reduce to the cr-form of the Painleve V equation. This 
reduction can be performed in the spirit of §7, following the corresponding part of 
[JM, Appendix C]. Although we do not perform the computation here, let us state 
the result. 

Theorem 8.10 [Tr]. Assume that s > 0. Then the function 

dlndet(l -K|(s,+oo)) 
= ' ds 

satisfies the differential equation 

{sa"f = (2(f7')^-S(T' + (7+(zyi+Zy2 + J^3 + J^4)(T')^-4((7' + Zyi)((7' + Zy2)((T' + ZV3)(f7' + I/4), 

(8.2) 

where 

ui ^1^2 ^ 0, Us = -z, 1^4 = -z' . 
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This result can of course also be obtained from Theorem 7.1 via the limit tran- 
sition discussed in Proposition 8.8. 

Very much in the same way as the 2-^1 kernel becomes the Jacobi kernel at 
integral values of z and the Whittaker kernel becomes the Laguerre kernel if one 
of the parameters 2;, z' is an integer, see [P.III, Remark 2.4]. Without giving any 
details, we formulate the results which can be obtained using this specialization. 

Let J be a proper subset of ]R_|_ , whose left endpoint is allowed to coincide with 
0, and whose right endpoint is allowed to coincide with +00. 

Theorem 8.11. Assume that < a < ^. Then the Fredholm determinant det(l — 
-^n^^l j); where K]f-^ is the nth Laguerre kernel with parameter a, is the t -function 
for the system of Schlesinger equations (8.1). The matrices {Cj}^^i are nilpotent 
{if a I — or a2m — +00 then Ci — or C2m — 0, respectively) , and the eigenvalues 
of A are equal to ±q;/2. 

Theorem 8.12 [TW4]. Assume that s > 0, ct > — 1. Let Kg he the nth Laguerre 
kernel with parameter a restricted to either (0, s) or (s, +00). Then the function 



a{s) 



dlndet{l-Ks) 
ds 



satisfies the differential equation (8.2) with i/i = 1/2 = 0, 1^3 = n, 1/4 = n + a. 

8.3. The confluent hypergeometric kernel. This subsection is based on the 
following observation. 

Proposition 8.13. Set 

z = zq + iT, z' = zo — iT, w = wq + iT, w' = wq — iT. (8.3) 
Then the 2F1 kernel has the following scaling limit: 



^(a^>y) = ^lim — , — , a:,y7^0, 

\x y J 

where the limit kernel depends on 1 complex parameter r = zq-\-wq, 3?r > — |, and 
has the form 



]C{x,y) = —T 



r + 1, r + 1 
23fJr + 1, 23?r + 2 



Q{x)P{y)-P{x)Q{y) 



r 
23?r 



x-y 

2ix 



r + 1 
2Kr + 2 



2ix 



Here iFi ^ a; is the confluent hypergeometric function also denoted as $(a, c; x), 
see [Er, 6.1]. 

The determinantal point process with the correlation kernel }C{x, y) describes the 
decomposition of a remarkable family of measures on infinite Hermitian matrices 
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on the ergodic (with respect to the U{oo) action) measures, see [B04]. We will call 
]C{x,y) the confluent hypergeometric kernel. 

For real values of r this kernel was obtained in [WF] as a scaling limit of 
Christoffel-Darboux kernels for a certain system of orthogonal polynomials (called 
the pseudo-Jacobi polynomials). For complex values of r such limit transition can 
be carried out as well, see [B04, §2] . 

Proof of Proposition 8.13. This is a direct computation. The relevant limit relation 
for the hypergeometric functions in this case, has the form 



The determinantal point process defined by K, has locally finite point configura- 
tions almost surely, see [B04]. Hence ([So, Theorem 4]), the restriction Kt = ^|(o,t) 
to any finite interval (0, t) defines an operator of trace class, and det(l — ICt) is 
well-defined. It is natural to conjecture that this Predholm determinant satisfies 
a differential equation obtained by taking the corresponding scaling limit of the 
Painleve VI equation of Theorem 7.1. 

In the Proposition below we check that the limit of the differential equation 
exists, and we observe that it is a a-form of the Painleve V equation. In [WF] it 
was proved that for the real values of r the determinant det(l — /Ct) does indeed 
satisfy this equation. The justification of this statement for all values of r in our 
setup requires a proof that the corresponding restriction of the 2F1 kernel converges 
to JCt in trace norm, and we leave this technical issue aside here. 

Proposition 8.14. Under the change of parameters (8.3), and the change of the 
independent variable s — T/t, equation (7.1) converges to the following a-version 
of the Painleve V: 



-{ta"f - {2{ta' - 5) + (?')' + - r)a'f - {a'f{a' - 2ir){a' + 2if), (8.4) 



Proof. We derive (8.4) from (7.1), assuming that certain limits exist, as noted 
above. Keeping in mind the relation s = T/t, we have 




□ 



where r = zq + wq and 






+ 0(1) as T — > +00, we define 
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which leads to 

da{s) _t^d^rT /^^^ i{r-f) ^ ^ {r + f)^ \\_^d^{t) ^^^^ {r + 



ds r dt \ t \ ^ ' 2 4 / / dt ' ' 4 



s 



d(r{s) ^ f dajt) i{r-r) \ 
-^-""^'^^^[-li 2— j 



d^a{s) e d f da{t) „ , , (r + f)2\ d''a{t) 



ds^ T dt dt ""^^^ 4 J T dt^ ' 

Substituting these relations into the equation (7.1) and passing to the limit T 
+00 we obtain 



^ dt ' ' 4 J \ dt^ 
= I o/^ ^5^(0 z(r-f) \ r^dajt) (r + f)^ \ (r + f)^ (r - f) ' 

'Vci* 2yVc^i 4y 4 



^ ^ 'J \ dt ' ' 4 4 

It is readily seen that the right-hand side of this equation is divisible by 

da{t) {r + ff 

and after this cancellation the equation exactly coincides with (8.4). □ 
Remark 8.15. 

For r — the kernel /C becomes the sine kernel sin(a; — y)/7r{x — y) , see [B04, 
§2]. Accordingly, equation (8.4) takes the form 

-{ta"f ^ 4{ta' -a){ta' - a + (a'f). (8.5) 

This agrees with the celebrated result of [JMMS], which states that equation (8.5) 
is satisfied by the function 



_ , , <i , , / , sin(a; — y) 
a{t) =t — Indet 1 ^ 



dt \ Tr{x — y) 



(o,t), 



9. Differential equations: a general approach 

Lemmas 4.3, 4.4 point to a general method for proving that a wide class of 
determinants satisfy Painleve equations (see the Introduction and [TWl-4]). We 
illustrate the method in the case of the Airy kernel 

Ai{x)Ai\y)-Ai{x)Ai\y) 

A{x,y) = 

x-y 
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where Ai(x) is the well-known Airy function. This kernel arises in random matrix 
theory ( [F] , [TW2] ) and plays a central role in the interaction of combinatorics and 
random matrix theory (see e.g. [BDJl], [BDJ2], [Ok], [BOO], [J]). 

For s G M, let Ag denote the operator obtained by restricting the kernel A{x,y) 
to L2(s, +oo). The basic result of Tracy and Widom [TW2] is that 

-— lndet(l - A,) ^u^{s), 

where u{s) solves the Painleve II equation 

u" = 2u^ + su 



with initial conditions 



u{s) ~ —Ai{s) as s ^ +00. 



We now outline a proof of this fact using Lemmas 4.3, 4.4. It will be clear to 
the reader that the method extends, in particular, to the general class of kernels 
considered in [TW4]. 

In the notation of Lemmas 4.3, 4.4, let S = M, E2 = J = (s,-|-cx3), and Ei = 
E \ E2 = (— oo,s]. Let -B(C) be a 2 x 2 fundamental solution of the differential 
equation 



dB{C) 



C 

1 



B{0, deti?(C) = l, 



with Sn(C) = ^^'(0, S2i(C) = ^^(0- Set 



m(C) = 



-27Ti 

1 



Then m satisfy the jump relation m+{x) = m-{x)v{x), a; e R, where 

v{x) 



1 27Ti 

1 



Set 



X e 



and note that v = I + 27rifg*. Also 



F(x) = m{x)f{x) 
and we set 



Ai/{x) 
Ai{x) 



G{x) = m *{x)g{x) 



-Ai{x) 
Ai'{x) 



V2{x) = I - 2'KiF{x)G\x), X eT,2 = {s, +00). 
Let nis solve the normalized RHP (S2,V2), "7.5 (C) — > / as C — oo- By Lemma 4.4, 



M{0 = m,{C)m{C) 
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solves the "simple" jump relation M+ = M_v on Ei. Standard arguments as in 
Theorem 5.1 and Proposition 6.2 now imply that M satisfies the Lax pair 



a C + b 
1 -1 



+ 



c 



P Q 
r —p 



M(C), 



ds 



1 



p q 
r —p 



M(C), 



(9.1) 
(9.2) 



where 



p q 
r —p 
only on s. By (6.9), 



is nilpotent. Here a, b, p, q, r are suitable constants which depend 



d 



— lndet(l - As) = iH-\s)H'{s))2i, 
ds 



where the prime refers to the derivative with respect to (, and 

M(C) = H{C) 



1 27rzln(C-s) 
1 



(9.3) 



As noted, in §4 (see (4.3)), detH{Q = 1 and H{Q is analytic near C = s (in fact, 
H{C) is entire). Using (9.1) and (9.3), we find 



which leads to the relation 



H 



1 




a C + b 
1 -1 



+ 



1 



p q 
r —p 



{H-\s)H'{s))2i = 2ap+q+{s + b)r. 
The compatibility of the Lax-pair equations (9.1), (9.2) yields ([KH, §2]) the rela- 



tions 2ap + q + {s + b)r — a. 



da 
ds 



r, where r solves the Painleve 34 equation 



d'^r 1 f dr 



ds'^ 2r V ds 



4r + 2sr. 



Writing r = —u^ , a simple calculation shows that u solves the Painleve II equation. 
This verifies the above claim for 

, , /-. ^ X da 9 
— lndet(l -As) = — - = -r = . 
ds^ ds 

Note that one can also show that det(l — Ag) is the r- function for the isomon- 
odromy deformation described by (9.1), (9.2). 

Appendix. Integrable operators and Riemann-Hilbert problems 

This appendix contains a brief summary of results on integrable operators and 
corresponding Riemann-Hilbert problems which can be found in [IIKS] , [KBI] , [De] . 
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Let S be an oriented contour in C. We call an operator L acting in L^(E, \dC,\) 
integrable if its kernel has the form 



£(C,C')= ^'"'/y'^'' , C.C'eS, 

for some functions fj, gj, j — 1, . . . , N. We shall always assume that 

N 

so that the kernel L is nonsingular (this assumption is not necessary for the 
general theory). 

We do not impose here any restrictions on the functions fi,gi and on the contour 
S. For our purposes it suffices to assume that E is a finite union of disjoint (possibly 
infinite) intervals on the real line, fi,gi are smooth functions inside &, and 

e Lf(S,|dC|)nL°°(S, |cZC|) forsome p, l<p<+oo. (A.l) 

These restrictions guarantee, in particular, that L is a bounded operator in -Zv^(S). 

Particular examples of integrable operators appeared in the mathematical phy- 
sics literature a long time ago. However, integrable operators were first singled out 
as a distinguished class in [IIKS]. 

It turns out that for an integrable operator L such that (1 + L)~^ exists, the 
operator K = L(l + L)~^ is also integrable. 

Proposition A.l [IIKS]. Let L be an integrable operator as described above and 
K = L{1 + L)-^ Then the kernel K{C, (') has the form 

where 

F, = (l + L)-V,-, G, = {l + L')-'gj, j = l,...,N. 

^/Ef=i fjiOgjiO = on E, then Y.f=i Fj{C)GjiO = Oon^as well. 

A remarkable fact is that Fj and Gj can be expressed through a solution of an 
associated Riemann-Hilbert problem (RHP, for short). 

Let V he a map from S to Mat(/c, C), where /c is a fixed integer. 

We say that a matrix function m : C \ S — > Mat(/c, C) is a solution of the RHP 
(S, v) if the following conditions are satisfied 

• m{() is analytic in C \ E, 

• m+(C) = m_(C)w(C), C e E, where m±(C) = lim to(C'), 

C'G(±)-side 

If in addition 

• m{() — >• / as ^ — >• CX3, 

we say that m solve the normalized RHP (E, v). 
The matrix v{() is called the jump matrix. 
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Proposition A. 2 [IIKS]. Let L be an integrable operator as described above such 
that the operator 1 + L is invertible. Then there exists a unique solution m{() of 
the normalized RHP (E, v) with 



v{C) =I + 27ri fiOgiCY e Mat(iV, C), 



where 




9 = (fi'i, • • • ,5'^)* , 
has the form 



C, C 6 s, 



where 



F={F^,...,Fm)\ 



G — {Gi, . . . , Gn)* 



are given by 



F(C) = m+(C) /(C) = m_(C) /(C), G(C) = m+\0 9{C) = mZ\0 9{0- 



In other words, the inverse (1 + L)~^ of 1 plus an integrable operator can be 
expressed in terms of the solution of an associated problem in complex variables. 

The function m(C) may have singularities at the points of discontinuity of the 
jump matrix v (e.g., at the endpoints of ©). Unless specified otherwise, we assume 
that m(C) belongs to the L^-space locally on any smooth curve passing through 
the singular point. Under our restrictions on fi,gi, and &, see above, the solution 
m(C) in Proposition A. 2 satisfies this condition. 

A discrete version of the Propositions A.l and A. 2 is given in [Bor2]. 

Let now S = S/ U S// be a union of two contours. Assume that the operator L 
in the block form corresponding to this splitting is as follows 



L{x,y) 





hi{y)hii{x) 
x-y 



hi{x)hii{y) 
x-y 







for some functions /i/( • ) and hii{ ■ ) defined on S/ and S//, respectively. 
Then the operator L is integrable with N = 2. Indeed, 



L{x,y) 



fi{x)gi{y) + f2{x)g2{y) 



x,y eT,, 



x-y 



where 




X e S/, 
X e S/7, 




X e S/, 
X e S//. 



The jump matrix v{x) of the corresponding RHP has the form 
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It can be easily seen that the RHP in such a situation is equivalent to the following 
set of conditions: 

• matrix elements mn and m2i are holomorphic in C \ E/j; 

• matrix elements mi2 and m22 are holomorphic in C \ E/; 

• on E77 the following relations hold 

mii_|_(x) — mii_(x) = 2ni h'j(x)mi2{x), 
m2i + (x) - m2i-{x) = 2ni hj{x)m22{x); 

• on E/ the following relations hold 

mi2_|_(x) — mi2-{x) = 2Txi h'jj{x)mii{x), 
'm22+{x) - m22-{x) = 2ni hj j{x) 17121 (x); 



• m{x) ~ / as X ^ 00. 

According to Proposition A. 2, the kernel K(x, y) in block form corresponding to 
the splitting E = E/ U E// is given by 



K{x,y) 



hi{x)hi{y)(—mii{x)m2i {y)+m2i (x)mii (y)) hj {x)hj j {y){mii{x)m22{y)—'m2i{x)mi2iy)) 
x—y x—y 

hii{x)hi{y){m22{x)mii{y)—mx2{x)m2i{y)) hii{x)hii(y){—m22(x)mi2{y)+m,i2{x)m22(y)) 
x—y x—y 

(A.2) 
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